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Editorial 

As behoves the annual publication of the Network which supports community-oriented 
education, the first section of this Volume presents twelve papers that focus on this topic. 
We have attempted to group these papers for the convenience of the reader. Thus, the 
Eleven Steps by Magzoub and colleagues and the Model of Experiential Community- 
based Education by Jinadu represent important contributions towards the development of 
a more structured approach and an underpinning philosophy. This is balanced by Magzoub 
exploring the problem of Network graduates in the face of health systems that positively 
discourage the practice of what they have learned. Omotara and colleagues expand this 
view of the real world with an extensive evaluation of attitudes towards rural posting and 
specialty preferences. 

The next three reports on experiences form Nigerian universities arc followed by a further 
paper on how staff from a well established school assist a new medical school in planning 
a careful preparation of staff, students, authorities and community leaders for the 
community-based experience. By contract, the paper by Landis and colleagues reports a 
North American programme for exposing students to community-oriented care. Christiansen 
and colleagues, also in North America, show how community-based education can be 
planned, organised and implemented by practitioners in the community. 
This section of the Annals reaches a high point, as it were, with a further paper by Magzoub 
in which he demonstrates triumphantly just how effectively a medical school in Sudan can 
affect change in the local health system, so that it really does provide opportunities for 
promotive and preventive, as well as curative care. This, too, is the message of the last 
paper which reports on a special course for district medical officers in Senegal. 

The next ten papers concentrate on evaluation. The editors have followed the advice of a 
member of the editorial board that reporting on aims, goals, objectives and curricula might 
change towards more robust exploration of methods and outcomes. The Annals are indeed 
fortunate to have attracted the collaboration of a veritable galaxy of experienced col- 
leagues. It may not be too far fetched to anticipate that this collection of papers will become 
the reference point for a growing volume of research and investigations into what we plan, 
what we execute and what we achieve in the education of health professionals. The paper 
by Mennin and his colleagues and the paper by Rotem range over a number of issues that 
set the scene, as it were. Graham and Kings open up a vista that offers a far reaching 
challenge to those who see evaluation as an essential means towards bringing change 
about 

Bandarany ake with his colleagues reports on a multidimensional approach to the evaluation 
of change from a five year to a six year curriculum in a well established medical school in 
Australia. Charting the Winds of Change is reproduced here, as it is almost certainly the 
most substantial paper from the 1980s on evaluating innovative curricula. This paper is 
followed by thoughtful up to date reflections on the same theme by Woodward in Canada. 
The next two papers from the Netherlands by Schmidt and colleagues and by Majoor 
respectively represent outstanding examples of open and unequivocal reporting. Both 
papers will render valuable service in stimulating debate on how problem-based learning 
can be made even more acceptable and effective by ensuring that the wider educational 
environment is truly supportive in the overall context of helping late adolescent lay men 
and women to mature into young adult professionals. 
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The last two papers in this section relate to the evaluation of teachers. The literature review 
by Wakefield and her colleagues identified one paper only that reported on the evaluation 
of problem-based, small group tutors. However, the authors were able to validate tkf ir own 
instrument against information from colleagues at other like minded schools in North 
America. The paper by Zhivkova, by contrast, sets out to establish staff and student 
originated criteria for evaluation of conventional lecturers 
in a traditional system. 

The section on innovation and change includes three important papers that might well 
have deserved pride of place at the very beginning of this Volume. From New Zealand the 
Annals are fortunate to be able to print the well documented account by Lewis and her 
colleagues of how a traditional clinical course was progressively changed into a problem- 
based course. In addition, be it noted, these busy clinicals found time and energy to satisfy 
their intellectual curiosity to find out the extent to which the new way of learning affected 
the learning style of the students. Des M archais and his two colleagues have given us one 
more important paper from Canada, how Sherbrooke extended its educational revolution 
throughout its clinical curriculum. Notable are the philosophy on which the change has 
been based and the sensibility which has supported the staff in achieving the change. The 
third paper comes from Great Britain, where Towle illustrates how a charitable trust uses 
its neutral credibility and materially assists in changing a climate of opinion - an essential 
prerequisite for change of any real magnitude. The fourth paper by Hartono and colleagues 
reports an introductory course to the clinical years, with emphasis on role play and 
experiential learning. 

Probiem-based learning specifically is represented by two explicit contributions. Snel- 
len-Balendong has given us an interesting insight into the Maastricht tutorial approach and 
the materials used for this experience. Eagle and colleagues have investigated what 
students identify as their learning goals during such tutorial sessions. These authors have 
compared the students 9 own learning issues with those intended by the author of the 
learning problem. The paper illustrates just how much remains to be explored about the 
process of problem-based learning. 

Administration of education, the last section, is an aspect that tends to attract less 
attention than it deserves. We are, therefore, very fortunate to be able to publish the account 
by Areskog and his colleagues of their three year experimental programme of student 
selection in Sweden. Norman has contributed th? last and equally important paper which 
presents cogent arguments for placing the responsibility for active planning and 
experimentation in medical education in the hands of the academic staff, rather than relying 
on a unit of expert educationalists. 

As in the past, this Volume too concludes with information about recent publications and 
a selection of abstracts froru the literature. Let the inordinate length of this editorial be 
justified by the extraordinary wealth of the contributions which the Annals has heen 
privileged to publish for the benefit of the membership and for the growing number of 
readers in institutions which are not yet part of the Network. 
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COMMUNITY-ORIENTED EDUCATION 

Eleven Steps of Community-Based Education »£ Gezira Medical 
School 

Mohi Eldin M. A. Magzoub, B. O. Ahmed and S. T. Salih 
University of Gezira 
Wad Medani, Sudan 

Abstract 

This paper suggests clear steps in community-based learning ( CBL) as in problem-based 
learning. These steps follow the management cycle or problem solving cycl The steps 
include selection of a community unit, identification of investigation tools, data collection, 
analysis, listing of problems, prioritisation of problems, plan of action, implementation of 
the plan, follow up, evaluation and reporting. The features of these steps and how they are 
applied by the Gezira Medical School are discussed 

Introduction 

A major objective of medical schools is to prepare their students to resolve the problems 
of their communities at the level of individuals, families and the community. In addition, 
the students have to be prepared to continue their education following graduation to 
become life long learners. The realization of these goals has been the subject of numerous 
initiatives and innovations. Now many innovative schools follow the clear steps of 
problem-based learning as described in the nine steps (Schmidt, 1983). There is a need to 
create similarly clear steps for community-based learning (CBL). 
Community-oriented medical education (COME) is defined as "education which is 
focused on population groups and individual persons taking into account the health needs 
of the community concerned" (Network of Community-Oriented Educational Institutions 
for Health Sciences, First Meeting, 1979). Community-based education "is the means by 
which COME can be achieved". Community-Based Leanang "is the specific learning 
activities in community settings" (Hamad, 1991). From these definitions we conclude that 
CBL is learning activities that use individuals, families and communities as units for 
learning. Our steps in CBL exclude individuals because here the problem solving or 
problem-based strategies arc applied. However, the aggregate data of individuals attending 
primary health units, for instance, will constitute a major resource in our steps for CBL. 
Our focus is thus on CBL using groups of individuals in primary health units, families and 
communities. Care of these groups calls for special competences that students have to 
acquire, in order to become community-oriented problem solvers. 

Request for reprints should be sent to Mohi Eldin M.A. Magzoub, University of Gezira, Faculty of Medicine, 
P.O.B. 20. Wad Medani, Sudan. 
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The Need for Clear Steps in CBL 

As more educational institutions come to adopt CBL as a main educational strategy, clear 
steps will facilitate the sharing of experiences which will in turn be reflected in the further 
development and progress of CBL. Many institutions involved in CBL have similar 
community-based programmes e.g.. Gezira, Suez Canal, florin, Aga Khan Universities. 
This will make it easier to agree on uniform steps in CBL. One of the criticism of COME 
and CBL is that they are based on "soft science" and not on "scientific grounds'* (Hamad, 
1982). The defining of these steps may help in conceptualising CBL in a uniform 
framework which may contribute to making CBL mote scientifically based. Specifying the 
steps will define students* learning objectives which should facilitate teaching and 
learning activities and contribute to more objective assessment of students and evaluation 
of programmes. 

Main Features of the Proposed Steps 

These steps follow the management cycle or problem solving approach, where students 
progress through different stages, including planning, implementation and evaluation, lie 
students are exposed to real and practical community field experiences. In contrast to 
problem-based learning in the class room, with its heavy reliance on written and simulated 
problems, the students are offered the chance of real life exposure to community problems 
and to apply their knowledge and skills. As community participation is an integral part of 
the educational process, it is not specified as a separate step. Most of the steps are followed 
by provisional report writing; however the final report is listed as the final step as important 
in its own right and because it includes many learning objectives. An important feature of 
these steps is that it is not always essential to go through all the steps for every problem. 
Selection of steps and their sequences could be adapted according to the available 
resources, time and programme objectives. 

Step One. Selection of Community Units (Family or Community) 
In most instances the selection is the responsibility of the Medical School. This selection 
is determined by logistics, access to transport, programme objectives and the willingness 
of the community to accept students and to participate actively in the programme. A variety 
of community units will be selected in terms of development, size, ethnic and geographical 
background, so that the students can share experiences between and within different 
groups. At this stage the learning objectives include introduction to community and family 
structure and function, and the impact on the health of individuals, the importance of a 
multidisciplinary approach and inter-sectorial collaboration in health. 

Step Two. Identification of Investigation Tools 

The students are expected to identify and design all the tools of investigation for collecting 
data from the community units. Students work independently in small groups under the 
supervision of staff members. The learning objectives here are to recognize, apply and 
evaluate the strength and weaknesses of each instrument, and the data collection which 
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may include questionnaires, direct interviews with community leaders, and observations. 
Also at this stage the students will have the chance to use and advance their knowledge of 
sampling procedures and of the conduct of surveys. 

Step Three. Collection of Data 

The students collect data from their community units during one or several visits. The 
students will be offered the chance to take full responsibility in approaching the community 
leaders and arranging their stay in the community. The learning objectives are: to be able 
to administer the data collection instruments, to use and develop their knowledge and skills 
in communication, how to make use of community participation and to become acquainted 
with rural life and the difficulties and constraints associated with projects in the field. 

Step Four. Analysis of Information 

At this stage the students come to the university campus to analyze and interpret the data 
they have collected. They work individually or in small groups with the assistance of their 
tutors, the library and senior peers. By the end of this step the students are expected to be 
able to recognise all the stages of data processing which include coding, entry, cleaning, 
manipulation, analysis and presentation. The students will also have the opportunity of 
applying their knowledge of statistics and to learn more about statistical procedures. The 
students are introduced to the use of computers in data analysis. 

Step Five. Listing the Problems 

The students are asked to list the problems and classify them in a presentable format, e.g. 
according to problems of health status, health service problems and community problems. 
The learning objectives are: to recognise basic feature of a problem, problem classification 
and presentation. 

Step Six. Prioritisation of Problems 

The students list the problems in order of priority and follow their priority problem through 
the subsequent steps. Also at this stage the students include in their report general 
recommendations about all the problems according to their findings. The criteria for 
selecting a priority problem may include frequency, seriousness, preventability and the 
community's perception of the problem. Ourexperience has shown that the most important 
criterion is the community* s perception of the problem. This encourages active community 
participation and thus the proper implementation of the intervention. 

Step Seven. Plan of Action 

The students write a detailed plan of action for the resolution of the problem. During this 
process the students will be helped by their tutors. The plan will be taken to the community 
for discussion and final approval. With the help of the community, the students will 
approach funding agencies to finance their suggested interventions. The learning objectives 
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here are: how to write a project proposal including objectives, strategies, implementation 
planning, follow up and evaluation. How to approach funding agencies. 

Step Eight Implementation of the Plan 

This is the implementation of the suggested plan of action, with as much participation by, 
and financial support as possible from the community. This is the only way to ensure 
sustainability of the project. Here the students will learn implementation with a real 
experience of working with villagers through their participation. 

Step Nine. Follow-up 

Students follow their project up on a regular basis, either through regular visits or meeting 
community leaders during their visits to the town. Students use a checklist to follow their 
activities. The learning objectives arc: to acquaint students with the importance of 
supervision and supervisory skills. The results of supervision will be used for decision 
making and evaluation. 

Step Ten. Evaluation 

This is where the students evaluate their project by using different evaluation tools to 
determine the degree of achievement of the objectives of the project. The learning 
objectives here arc: to recognise and apply different tools of evaluation, assess the 
advantages and disadvantages of these tools, and use the results of evaluation to inform 
decision making. 

Step Eleven. Reporting 

Reporting may accompany all the above steps. However, the final report is the most 
important one. Here, the students describe the whole process including all the steps. This 
report will be discussed with the community. During this step the students learn how to 
write a scientific report. 

Application of these Steps at Gezira 

These steps are applied gradually and built on throughout the undergraduate course. 
Application starts in the first introductory part of the curriculum, the introduction to 
medicine course. Here the students arc given community problems and asked to analyze 
them by using existing records, making visits, and conducting simple interviews. This 
exercise includes Steps 1 to 7. Subsequently the students experience all the steps in depth 
under close supervision by the staff. This fits in with the interdisciplinary field training 
research, rural development and the family health courses. Later on the students have more 
autonomy and responsibility and less support from the staff. Here the students arc left free 
to choose the community units themselves and tailor the steps and sequence to the problem 
which they have selected. This is part of the rural residency course. 
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school 
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An Example 

Wc are a group of four students. Abbara village was selected for us during the summer field 
training. Our first task was to identify problems in this village. We decided to collect the 
data through a structured questionnaire, interviews with community leaders, and an 
observation checklist These tools were developed with the help of our tutors, senior 
colleagues and colleagues from the Faculties of Agriculture and Economics. It took us one 
week to develop these tools. It was hard work but it was interesting. We used public 
transport to reach the village. In the bus we met many people from the village, who 
introduced us to the Sheikh, the village chief. We had a warm welcome from the Sheikh, 
he asked his son and one of the few educated ladies to be our guide during our one week 
stay in the village. We lived in the Sheikh* s house which was a wonderful experience. We 
accomplished the task of data collection successfully. We returned to the University main 
campusfor data analysis and listed all health and health related problems of the village. The 
findings were discussed with our tutors, colleagues and the community leaders. With the 
criteria for prioritizing problems we selected childhood diarrhoea as our priority problem. 
In the second phase of our community-based education we chose a latrine construction 
programme as an intervention to address this priority problem. The objective was to 
increase the availability of latrines by 30% in one year. We submitted a joint project with 
the village health committee to the Blue Nile Health Project. They agreed to provide slabs 
for any completed four meter depth latrine. Our role was to encourage families to dig their 
four meter pit. It was difficult to convince families to change their behaviour, i.e. not to get 
rid of their excreta in the open air. Through time and personal contact we convinced the 
Sheikh to make a start. In far less time than we had expected our target was achieved, and 
the programme was evaluated in the last phase. There was a significant reduction in the 
incidence of diarrhoea when compared with the base line data. During the presentation of 
our work, we were criticised, as the reduction of diarrhoea incidence might not have been 
due to our intervention. It might have been the result of other factors which w had not 
controlled, including improved socioeconomic status and water supply. At first this was 
a disappointing comment but it has stimulated us to read more about study design and the 
control of confounding variables. We concluded that we could not be sure that the 
reduction was due to our intervention. What we can be sure about is that the community 
and we ourselves were satisfied with our inputs, and we learned a lot from this experience. 
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Model of Experiential Community-Based Education for Health Care 
Professionals: Implications for Developing Countries 

Musa KL Jinadu 

Obafemi Awolowo University 

De-Ife, Nigeria 

Abstract 

The relevance of traditional education of health professionals to the essential health needs 
of the community has been a subject of intensive debate world-wide. Experiential 
community-based education is a unique way of educating health professionals towards 
achieving this relevance. This model requires active gathering of information necessary 
to solve community health problems, encourages initiative, responsibility, cooperation, 
exploration and independent learning. Experience in the application of the model to PHC 
education at the Ife Medical school is discussed. The paper concludes by highlighting the 
implications of the model for PHC education in developing countries. 

Introduction 

Over the past two decades there has been a tremendous increase in the number of health 
workers trained in Africa. For example, between 1979 to 1986 approximately 21,900 
community health workers were trained exclusively for the primary health care (PHC) 
programme of Nigeria (Jinadu et al., 1990). A considerable number of physicians and 
nurses have also been trained. Despite this heavy infusion of manpower into the health care 
delivery system of Nigeria and other African countriesn there has been no corresponding 
improvement in the health status of the people. Infant and maternal mortality rates 
remained unacceptably high - approximately 120/1000 live births and 16/1000 deliveries 
respectively (United Nations Children's Fund, 1988). Diseases, such as malnutrition and 
infections, which can be controlled or eradicated through PHC programmes and strategies 
were the major causes of high morbidity and mortality in Africa (United Nations 
Children's Fund, 1988; Daramola, 1978). 

In the past, training programmes in Africa produced doctors and nurses who could serve 
effectively only within tertiary and secondary care institutions, leaving the PHC level 
unserved or underserved. 

Basic medical and nursing education in African countries was patterned on the British 
model which was entirely hospital-based. It also developed its power base in hospital and 
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medical school establishments where clinicians were the high status group. The growth of 
hospital medicine led to increased specialisation and sub-specialisation, and, consequently, 
to fragmentation and lack of coordination of care among the health professionals (Jinadu, 
In press). 

Following growing criticism of this medical education in Nigeria and the West African 
sub-region, the Nigerian Medical Education Working Party made a number of observations 
and recommendations in 1977 for the improvement of medical education in Nigeria. The 
Working Party recognised the need to make the education of doctors and other health 
professionals relevant to the needs of the population. A chapter of the report was, therefore, 
devoted to the Federal Government Health Policy of the Third Development Plan, known 
as the Basic Health Scheme, and the working Party recommended "a rural posting, in order 
to enable students to render community service" and "study the totality of the patient in his 
environment" (Fagin, 1978). 

Community-Based Education 

One of the main objectives of nearly all the medical schools in Nigeria was to train doctors 
with a strong inclination towards broad community care and preventive medicine. 
However, how this laudable educational objective should be achieved has not been 
adequately specified by nearly all the schools. Experience has shown that in some medical 
schools, planning, implementation and even evaluation of this aspect of the educational 
programme have tended to be haphazard, uncoordinated and ineffective. This has resulted 
in poor student motivation to achieve the objectives of community-based care, especially 
in the area of PHC programmes. 

Health is a total community process and the whole community structure must, therefore, 
be considered when dealing with health problems. PHC programmes and approaches are 
based on this premise. Effectively organized PHC orientated educational programmes for 
students should serve as a major factor for increasing the interest of students in PHC 
(Medical Education Working Party, 1977; Richards, 1986). 

Community-based education should be problem-based and student-focused (Medical 
Education Working Party, 1977; Schmidt et al., 1987). The problem-based, self-directed 
learning approaches are based on the premise that when students are presented with real 
life health problems, especially in the community, rather than the bits and pieces of fact- 
learning, they are highly motivated to acquiring the skills necessary for solving the 
problems (Coles, 1985). 

Model of Experiential Learning 

A model of an experiential programme represents a unique approach for achieving the 
objectives of PHC education for health professionals. The model presents us with a rational 
and systematic approach to community-based education. The focus of the model is on 
learning rather than teaching and training. In the model learning experience are conciously 
and systematically organised for active participation of the students in the tasks that are to 
be mastered. The teacher's major role is to select appropriate learning experiences for the 
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development of problem-solving skills, attitudes and interest for the achievement of the 
predetermined objectives. 

Table I contrasts traditional health professional education with the experiential model for 
a) sources of information available for learning; b) the learning environment; c) the role 
of teachers; and d) the approach of the students to learning. 

Table I. Comparison of conventional and experiential community-based education 



Conventional Model 



a) Sources of 
information 



Experiential Model 



Authoritative source 
of information i.e. 
lecturers and textbooks 

Learning takes 
place in designated 
settings* e.g. lecture 
rooms 

Use of deductive and 
directive learning 
process 



Surface and strategic 
approach 



Observation and enquiry 



b) Learning 
environment 



c) Role of teachers 



d) Approach to 
learning 



Learning takes place 
in the entire social 
environment 



Use of inductive, 
non-directive learning 
process 



Deep level approach 



Information for active learning in the community-based, experiential jnodel is gathered by 
observation of the socio-cultural environment of the community, rather than by reliance 
on books and lectures. 

Learning, therefore, takes place in the entire social environment of the community as 
opposed to lecture rooms. The student is continuously exposed to circumstances that 
demand that he should adequately diagnose the situation, devise appropriate solutions and 
carry out remedial actions. 

Health projects in the community should, therefore, provide students with adequate 
exposure to the learning environment as a starting point to problem-solving. Information 
required for solving a community's health problems are gathered during this process. 
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In the experiential model, the role of the teachers is to help the students to think and work 
out problems for themselves using their own initiative* The teachers aid the students in an 
inductive learning process rather than the traditional deductive process (Medical Education 
Working Party, 1977). The model encourages the students to explore the meaning of what 
they encounter in the environment and, therefore, team work with other health professionals 
is also encouraged. 

PHC Education of Ife Medical Students 

The Faculty of Health Sciences of Ife University admitted its first medical students in 1 972. 
The institutional objectives of the Faculty include a clear commitment to train health 
professionals with a strong bias towards PHC. The Community Health Department of the 
Faculty of Health Sciences and its teaching hospital is mainly responsible for the PHC 
education of the Ife medical students. However, other members of the University, not only 
from the health sciences, but also from social sciences, education, and agriculture 
participate in this educational programme. The PHC programmes are based on health 
centres scattered throughout the Faculty's catchment area, which consist of about 500 
villages and two towns. Specific PHCfocused educational objectives that relate to the 
experiential model are: a) to orientate the students to community health problems of rural 
inhabitants; b) to enable the students to learn and put into practice the basic principles of 
community health diagnosis; c)to enable the students to initiate, plan, implement and 
evaluate programmes and activities for solving community health problems; and d) to 
develop team spirit for community health practice and PHC among the students. 

The students participate repeatedly in community based PHC activities during the 
preclinical and clinical years of their medical education (Table II). For example, during the 
preclinical years the students carry out field work related to community assessment This 
includes basic demographic and environmental surveys of the community. During the 
clinical years, the students rotate through health centres, where they are involved in various 
PHC programmes, cany out problem-solving research activities, and plan, conduct and 
evaluate interventional programmes. After some introductory lectures on the basic 
scientific principles of community health and PHC practice, they are encouraged to learn 
on their own. 

Assessment of Student Performance and Evaluation of Programmes 
Assessment of the students in this programme includes: 

1 . Assessment of the project report during seminar presentations and by independent 
assessors. 

2. In course and end of the course examinations. 

3. Attitudinal assessment of the students by their supervisors. 

4. Assessment of the degree of participation of the students by their supervisors. 
Each assessment is carefully weighted and scored to reflect the composite performance of 
the individual student. One of the most difficult aspects to assess objectively is each 
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Table n. Community-based education for Ife medical students 



Year 


Duration 


Activities 


Location and organisation 


YearH 

(Preclinical) 


4 weeks 
Block 

notfinc? 

UU9UIIK 


Demographic sur- 
vey of a community: 
design survey in- 
strument, man out 
the community, 
conduct the survey 
and write report 


Communities (urban and 
rural). 

Students are divided into 
small erouns 
Each group works in a 
village or community of 
about 5000 inhabitants. 


Year IV 
(Clinical) 


6 weeks 


Survey of specific 
health problems 
or programmes, 
community diagnosis. 


Communities/health centres. 
Group and individual activi- 
ties. 


Year VI 

(Final year) 


8 weeks 

Block 

posting 


Disease-oriented 
survey; planning 
and conducting 
intervention pro- 
grammes. 


Health centres/communities. 
Individual or group survey. 



student's contribution to a group project. V/e combined individual scores, obtained during 
seminar presentation of the project report, with the supervisors' scores, based on observation 
of the students during the project. 

Our approach to the evaluation of the model consists of end of posting review by the 
students and an annual departmental review of the programmes. During the students 9 
evaluation excercise, various aspects of the programmes are criticised by the students, and 
their criticisms are always taken into account during the review of the programmes. The 
annual review includes other members of the Faculty and University with shared interests 
in the programmes. The staff of the health centres, particularly the community health 
nurses, are especially welcome at the review, because they can give a clear picture of their 
community's perceptions of the students 9 programmes. 

Our experience shows that the model is widely accepted by students, academics and the 
community. The students have always been very enthusiastic to learn during their 
community-based activities, once their learning objectives were made explicit to them. 
Some academics were initially sceptical about the effectiveness of the model for the 
achievement of the objectives of community health education. They believed that certain 
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subjects had to be taught in-depth in the classroom - the conventional model - before they 
could be experienced. Experience with the experiential model has convinced them of its 
effectiveness. Adequate implementation of this model requires considerable input of 
resources in terms of transport, accommodation, time and effort Vehicles are needed to 
transport students and staff to and from the communities. Where the communities are far 
from the Faculty, accommodation has to be provided for the students and their supervisors. 
A great deal of time and effort have to be devoted to the supervision of the students by 
members of the Faculty. Members of the community are positive in their attitude towards 
the students and their activities in the community. Adequate care has always been taken 
to introduce the students properly to the communities, and the students are frequently seen 
providing services at the health centres. 

Conclusions 

In the absence of elaborate PHC facilities and programmes, especially in rural area, 
communities in developing countries should serve as a practice areas for PHC education 
of health professionals . Adequately planned and implemented community-based educational 
activities for the students require earlier clinical contact than is currently the practice in 
most educational institutions. The experiential model presented in this paper will provide 
the basis for developing skills in integration of knowledgde during the community-based 
activities and, therefore, help to avoid some of the pedagogical issues currently faced by 
medical education. Effective operation of the model will ensure that students do not 
disregard the PHC aspects of their curriculum. 

Behavioural objectives for the students as well as methods for student assessment should 
be well set out at the planning stage. 

Planning and implementation of the community-based educational programme should be 
a multidisciplinary activity, involving teachers not only from basic medical and clinical 
sciences but also from social, agricultural and educational sciences. Adequate planning of 
learning experiences based on the model will require sequential organisation and integration 
of situations in which the students- will encounter appropriate problems to solve. 
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Problems facing Graduates from Network Institutions: Experience 
from Sudan 

Mohi Eldin M. A. Magzoub 
University of Gezira 
Wad Medani, Sudan 

Abstract 

Graduates from the Gezira Medical School face many problems in the present health 
system. These problems include curative-orientated health care in both teaching hospitals 
and rural areas. Difficult working conditions which lead many graduates to emigrate. 
Postgraduate studies which are mainly adapted for international recognition rather than 
to meet reed health needs. These problems make the present health system unattractive for 
Gezira graduates and hinder their contribution in applying the competences which they 
learned in the undergraduate community-based curriculum. These experiences and how 
graduates from Gezira attempt to influence the health system are discussed. 

As a graduate from Gezira, a member of the Network, I am not only pleased but rather 
relieved to see the problems facing graduates as one of the top Network priorities for the 
next ten years. I think this issue is very vital and has to be considered seriously by the 
network schools if our schools are to play an influential role in the health services. We have 
to work jointly schools, graduates and ministries of health, to face this priority problem. 

Although this is a personal experience it is true for most Gezira graduates and probably for 
graduates of other Network schools, based on interviews and discussions with colleagues 
from Gezira. We blame traditional schools for focusing on irrelevant knowledge. Yet the 
same accusation might be raised in our case, that we provide our graduates with 
competences which they arc unlikely to be able to use in practice, in the present health 
system. 

Graduates from Gezira Medical School receive temporary registration from the Sudan 
Medical Council and join the Ministry of Health as house officers (residents). In Sudan this 
appointment is for one year during which the doctor has to spend three months in each of 
four departments: obstetrics, medicine, surgery, and an optional rotation in another 
department. All medical specialties are offered, except primary health care and public 
health. Indicators for evaluate of performance by doctors are totally different from what 
is taught at Gezira. The criteria include how quickly a diagnosis is reached and, more 
importantly, demonstration of sufficient knowledge to undertake postgraduate studies at 
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internationally recognised medical institutions. All young doctors compete for the 
opportunity to undertake postgraduate studies as soon as possible. The selection for 
postgraduate education is through a written examination in the clinical and basic sciences 
without regard to performance as a doctor. The fundamentals of training at Gezira are not 
at all relevant in such an environment Community-based activities are viewed as soft 
sciences, as side issues, and sometimes as a waste of time and a job to be performed by lazy 
doctors who failed to join attractive specialties. 

Another obstacle is the severe constraint imposed on quality of care by the shortage of 
doctors, where the house officer is involved in many responsibilities including working in 
the outpatient department (where patients stand in long queues), admissions, and inpatient 
follow-up. These duties create severe time constraints and work overload. In such a 
situation it is not possible for the young doctor to practise all the problem-solving skills, 
especially when it comes to the underlying problems at family and community levels. 

As new graduates we spent one year exclusively in big teaching hospitals without finding 
any real chance to practise the full range of competences we had learned in our medical 
school. By the end of the year we had succeeded in creating a name for our school, but 
according to their criteria and not ours. To give an example, during the drought we received 
a large number of displaced persons in the hospital. We suggested to the hospital authorities 
that an emergency clinic might be established at the camp to deal with all displaced persons 
at their camp. We were told that this was not our concern, that it was the job of somebody 
else in the public health office. We failed to do anything through the hospital. Eventually 
we made contacts through our school where we succeeded to involve students and 
established an emergency clinic to deal with the problems in the camp itself. Following our 
full registration by the Sudan Medical Council, many of us chose to work in rural hospitals. 
We thought that this would suit our training and the skills and knowledge we had acquired. 
To our surprise the situation in the rural hospitals was little better than in the teaching 
hospital. The health services were rendered independently through vertical programmes. 
The primary health units in the area were not under the supervision of the rurd hospital. 
They were supervised by the district inspector, located at the district headquarters. He had 
no time or facilities to supervise all the primary units in the district, usually more than 100 
units. This left the primary health units isolated and with many unresolved problems. 
Moreover, the financial and administrative responsibilities of the primary health units 
belonged to the local government councils, while the hospital remained under the Ministry 
of Health. Thiscreatedadichotomy inhealth services delivery. The community expectation 
of doctors is that they will apply high clinical skills and decide on few referrals, they are 
not expected to work in Primary Health Care and community based activities. 
The author was asked tc 'vork in a rural hospital in Eastern Gezira. Faced with all these 
difficulties, he organised a meeting with the community leaders to explain the importance 
of primary health care as a more appropriate way to face the health problems in the area. 
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He assured the leaders that this work would not be at the expense of his work at the hospital. 
It took a long time to convince the community. The author made contact with all the health 
workers in the area. Together with a colleague from Gezira, he started an integrated 
primary health care programme which included oral rehydration, nutrition and immunization. 
All the health workers in the area were trained in these interventions and followed this up 
with implementation. Evaluation demonstrated the success of the programme as measured 
by the health workers and through community feedback. Unfortunately, the programme 
was not strongly supported by Ministry of Health officials in the district and it could not 
operate within a supportive system; it was left to the enthusiasm of the doctor in charge. 
As was to be expected, the programme collapsed, due to the disinterest of the doctor ir 
charge. 

Another problem that faces all doctors in Sudan is low pay and difficult working 
conditions. This leads to severe drainage of doctors to the petrol rich countries in the 
Persian Gulf. Graduates from Gezira are not immune from these problems, and it is not 
surprising to find more than half the doctors of the first cohort have emigrated to Saudi 
Arabia. 

The existing postgraduate study programmes are mainly run by traditional medical schools 
with their own learning strategies and priorities. These programmes, imported from British 
institutions, are mainly adapted towards international recognition, rather than meeting 
community needs and local realities. There is no option for the Gezira graduate but to join 
these programmes with their own focus and orientation, which are very different from the 
graduates' basic training at Gezira. 

What is the Way out? 

The most important start for the Medical School is to play an influential role in the health 
services to ensure that the competences of its graduates are used appropriately. The Gezira 
experience will serve as an example. 

a. Health Area Policy 

As a result of meetings and discussions with the Ministry of Health an area health policy 
was developed. The aim of this policy was to deliver an integrated primary health care 
service led by the rural hospitals. Leadership was assumed by a health area management 
team (HAMT), headed by the doctor and including all Ministry of Health Departments, in 
addition to the related sectors. The team would be responsible for administration of the 
entire health services and personnel in the area for planning, implementation and 
evaluation of all health activities. The team would take charge of logistic support and drug 
delivery to all primary health units in the area. It would be responsible for the integration 
of all service*, including vertical programmes. It would seek active community participation 
where necessary. It was hoped that this system would solve the problems of the dichotomy 
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in health services delivery in rural areas and provide a healthy environment and hence 
attract Gezira graduates to play an influential role in the system. 

b. PHC Management Course 

This policy created a need for trained leaders to guide and manage health areas. 
Responding to this need the Gezira Medical School has organised a three week PHC 
management course for doctors prior to their enrolment in rural hospitals. The objective 
of this course was to prepare the doctor to perform the tasks of the health area management 
team leader. The methods of training followed the learning strategies of the GeziraMedical 
School, including problem-based and community-based learning. The course has been run 
1 7 times and has trained 420 participants so far. The training has had a magnificent impact 
on improving the quality of services at the rural hospitals as witnessed by its evaluation 
through site follow up visits. Moreover, the evaluation reports have shown that this training 
has demonstrated changes in the doctors' attitudes towards primary and rural health care. 

c. Influence of Gezira 

This course has shown that a medical school is able to play a very effective role in attracting 
to its philosophy doctors who are leaders of health services. . This, in turns, may influence 
the health services. The increasing number of doctors who are choosing public health for 
their postgraduate studies is attributed by Ministry of Health officials, among other 
reasons, to the influence of the Gezira course and the influence of Gezira graduates. 

The Gezira Medical School has started to influence the health services through its 
graduates who have taken leading positions in the Ministry of Health. One example is a 
graduate who is now working as an acting director of the National Control of Diarrhoeal 
Disease (CDD), a very important department in the Ministry. He caused quite a revolution 
in the CDD short training courses for doctors and other health workers by incorporating 
problem-based and community-based learning in these courses. That programme has also 
moved more towards community participation and involvement which had been neglected 
in the past. 

d. National House Officer Induction Course 

The Gezira Medical School, in collaboration with the Ministry of Health, has organised a 
o, e week orientation course for all new graduates in Sudan before they join the Ministry 
of Health as house officers. This one week course covers primary health car?, management 
and hospital administration. The learning and teaching strategies follow the Gezira 
curriculum. These short courses have proved that the school can play an important role 
through training of large numbers of young doctors who are potential leaders of health 
services in the near future. 

In conclusion the problems facing graduates of innovative schools of medical education 
arc numerous and may inhibit the influence of these schools through their graduates in the 



3m 



31 



health system. Fortunately the members of the Network are aware of the problem, and 
several innovative institutions have started to influence the health system. 
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Attitude to Community-Based Rural Posting and Specialty 
Preference 

B.A. Omotara, M.K.O. Padonu, A.P. Bassi and M.O. Amodu 
University of Maiduguri 
Maiduguri, Nigeria 

Abstract 

This prospective study is based on a questionnaire (Appendix) to six consecutive years of 
final year medical students (1986-1991) after they had returned from the 8 week 
community-based rural posting. The aim of the study was to assess the attitude of final year 
medical students to the rural posting and to explore the dynamics of specialty choice as 
a result of the posting. A total ofl 75 final year medical students were involved in the study 
consisting of 160 males and 15 females. Mean age was 25.5 years ranging from 21 to 35 
years. 

Result of the study showed that 54.7% of the students had decided on a specialty before 
their exposure to the rural posting, 16.6% changed their previous choice of specialty as 
a result of the posting, with slightly more than hcf of these changes (51.8%) in favour of 
Community Medicine. Of those who had not made a specialty decision before the posting 
13.6% reached a decision during or after the posting, and 44.4% of them decided to 
specialise in Community Medicine. Only one of the fifteen females decided on a specialty 
as the result of the posting, 7 did no make any decision before and after the posting, while 
7 had made decision before the posting and did not change their decision after the posting. 

72% of the students indicated that more than 80% of the * ^pectations were met during 
the end of the posting, while 28% indicated otherwise. All the students indicated that the 
rural posting changed their attitude favourably towards the practice of Community 
Medicine. 

Introduction 

The College of Medical Sciences, University of Maiduguri was established with the aim 
of producing competent doctors. The College was established in 1978, three years after the 
establishment of the University. The University was established purposely to provide 
qualified personnel for the then North Eastern part of Nigeria which now consists of five 
states; Borno, Yobe, Adamawa, Taraba and Bauchi. Before its establishment, this area 
benefited from graduates from Ahmadu Bello University, Zaria which is some 800 
kilometres away from Maiduguri. 
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Although the new College had made provision for rural posting in its curriculum, this 
posting was not immediately implemented because of logistic problems. Consequently the 
first three cohorts of medical students graduated from the medical school without 
undergoing therbnd posting. Instead, they undertook individual projects, though most of 
them were not community-based. In 1986, Song Local Government Area of Adamawa 
State, one of the University catchment areas was chosen for the 8 week community -based 
medical education programme . Because of the long distance from Maiduguri (approximately 
400 kilometres) and the shortage of staff in the Department of Community Medicine, it 
became impossible to continue to use Song Local Government for the rural posting after 
the second set of students in 1987. In 1988 arrangements were made with three other local 
government areas, Konduga, Bama and Gwoza. This arrangement was not implemented 
until 1989 and only in Konduga, the closest of the three Areas. From 1990 the posting in 
the three local government areas became fully operational. Gwoza, farther away, is only 
150 kilometres from Maiduguri, so thatFaculty staffed make daily trips for supervision 
when necessary, while the students were resident in the local government area. 

Prior to the eight week rural posting*, the medical students would have undertaken a six 
wed: non-residential community-based posting. This posting would normally come in the 
middle of the fourth year or second clinical year. The main purpose is to enable the students 
to acquire skills in data collection, analysis, and sampling and to appreciate the problems 
inherent in these procedures. The students are also introduced to school health services, 
and conduct visits to health related establishments, industries, both private and public. 
Reports of the visits are written and presented at the end of the posting. 

The subsequent eight week rural posting is designed to serve two purposes. It enables 
students to carry out rural community projects which they write up and present at the end 
of the posting. The students also provide clinical services in the rural health centres under 
the supervision of a medical officer. By the time they go to the rural posting, the students 
have completed their junior, intermediate and senior obstetrics and gynaecology and 
paediatrics experiences, as well as unior and intermediate medicine and surgery. The have 
also completed their laboratory medicine. It is hoped, therefore, that, by the end of the 
community-based rural posting the students will have appreciated the nature of their 
chosen profession, as well as acquired skills necessary to tackle health problems in the rural 
areas in which more than 70% of Nigeria's population resides. 
The purpose of this study was to assess the attitude of final year medical students toward 
the eight week rural posting and to determine how much exposure it gave them to the 
various specialties in the field of medicine, so as to enable them make a decision on a 
specialty. 

* See the paper on Community-Based Medical Education: The University of Maiduguri 
Experience" in this Volume of the Annals. 
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Method 

A 13 item semi-structured questionnaire (Appendix) was administered. The first part 
sought demographic information about the students* The second part concentrated cm 
information about die adequacy of the posting, aspects covered, adequacy of exposure to 
all specialties in medicine, how much of the students* expectation was met, problems 
encountered, and whether formal lectures should be continued curing the posting* The 
questionnaire was administered immediately after completion of the rural posting* Six 
cohorts of students were involved in the study (1986-91), with 160 male and IS female 
students* The questionnaires were collected immediately after completion* 

Results 

Table I shows the age and sex distribution of the 1 75 students who participated in the study* 
Hie total population of the students consisted of 160 males and IS females, with an age 
range of 21-32 years and a mean age of 2S*S years. 

Table I. Demographic information 

Age Group Frequency (%) 

20-24 66 (37*7) 

25-29 102 (58.3) 

30-35 7 (4.0) 

Male-160, female-15 students. Age Range 21-32 years, mean age 25*5 years* 



Table II summarizes the responses of the students* attitude to the rural posting* Most of the 
students (83*4%) agreed that the period of eight weeks was adequate for the posting* A 
majority (73*3%) felt they would like to be exposed to more than one local government area 
during the period of the eight weeks* A majority (86*9%) agreed that formal lectures or 
seminars should not be given during this posting* Many of the students agreed to have been 
adequately exposed to the specialties of paediatrics, medicine, and obstetrics and 
gynaecology, in that order, while exposure to surgery, and basic laboratory sciences was 
judged to have been inadequate* All the students agreed that the posting changed their 
attitude towards community medicine favourably* 

Table m indicates the areas where students encountered difficulties during their rural 
posting* The most problematic were transportation to their respective health centres and 
communities when they could not be accommodated within walking distance; and 
attitudes of the people* The most favourable areas were the attitude of the health workers 
and local government officials* 
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Tabic n. Summary of responses - rural posting and exposure to specialties 

Aspect Response 

Yes(%) No(%) 

Adequacy of the period of rural posting 146 (83.4) 29 (16.6) 

Restriction of niral posting to a 45 (25.7) 130 (74.3) 
particular rural area 

Formal lectures to be given during 32 (13.1) 152 (86.9) 
rural posting 

Adequate exposure to Surgery during 33 (18.9) 142 (81.1) 
the period of rural posting 

Adequate exposure to Medicine 129 (73.7) 46 (26.3) 
during the rural posting 

Adequate exposure to Paediatrics 140 (80.0) 35 (20.0) 

Adequate exposure to Obstetrics and 86 (49.1) 89 (50.9) 
Gynaecology 

Adequate exposure to laboratory sciences 26 (14.9) 149 (85.1) 

Changed attitude favourably toward 175 (100) NA. NA 
community medicine 



Table IV records the influence of the community-based rural posting on choice of 
specialty. Slightly more then half (53.7%) the students had already made a decision before 
the beginning of the rural posting; 29 (16.6%) changed their decision as a result of the 
posting, and 1 5 (5 1 .8%) of these changes were in favour of community medicine. Eleven 
(13.6%) made their decision during or after the posting, and five decided on community 
medicine. 
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Tabic HI. Difficulties experienced by students 



Problems Least Most 

Problematic (%) Problematic (%) 



Lack of adequate facilities 


7 


(4.0) 


31 


(17.7) 


Catering 


11 


(6.3) 


42 


(24.0) 


Transport 


13 


(7.4) 


20 


(11.4) 


Reference materials 


26 


(14.9) 


10 


(5.7) 


Attitudes of the people 


7 


(4.0) 


42 


(24.0) 


Attitudes of other health workers 


66 


(37.7) 


1 


(0.6) 


Attitudes of Local Government officials 


36 


(18.3) 


16 


(9.2) 



Discussion 

Studies of the attitude of medical students in Nigeria to the specialties of medicine and to 
rural medical practice have, in the past, suggested that factors other than medical school 
training influence or determine those choices (Erinoso, 1977;Brieger, 1979; Asuzu, 1989). 
For example, they showed that students, who grew up in rural communities, were more 
disposed towards rural practice and to primary care specialties than those who grew up in 
urban centres. 

The results of the present study show that community-based medical education can have 
a considerable impact on the attitude of medical students, because it gives them the 
opportunity to understand what the practice of community medicine really entails. 
Elsewhere students tend to complain about the abstract nature of community medicine, as 
they relate it to facts and figures rather than to clinical practice. The change of attitude 
towards community medicine that can be brought about by the exposure of students to a 
community-based rural posting is very significant in the sense that more than 70 per cent 
of Nigeria's population reside in rural areas, and it is more likely than not that many young 
graduates will find themselves in rural practice. The fact that the rural posting initiated a 
change of decision by those who had previously decided on another specialty is of 
significant importance. Omotara et al.» 1991) in a study of the dynamics of career and 
specialty choice with the same set of students showed that job satisfaction was one of the 
major factors influencing career choice and that further interesting experience/exposure in 
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a new specialty was a major factor influencing change from a previous choice of specialty. 
The results from the study coroborated thb conclusion, as further exposure to 
of community medicine initiated a change from a previous choice to community medicine. 
In the same study, unpleasant experiences/hazards in the previously elected specialty were 
the most important factor responsible for change from that specialty. In the present study 
no student decided to change from community medicine to another specialty. 

In addition, all the students who participated in this study indicated that their attitude to the 
practice of community medicine had been influenced more favourably by die eight week 
community-based rural posting. 

Now that primary Health Care (PHC) is the main focus and thrust of Nigeria's health care 
delivery system, community-based medical education occupies a central position in the 
curricula of all medical schools in the country. Two of die older medical schools in Nigeria 
anticipated this need much earlier and established community-based rural posting where 
students are adequately exposed to health problems in rural areas. The first was established 
in Igbo-Ora by the University of Ibadan Medical School some three decades ago, and the 
other i n Malumf ashi by the Ahmadu Bello University a few years later. Most of the second 
generation medical schools have fashioned their curricula along the same lines. The lack 
of participation and involvement by the community, indicated in this study, could have 
been due primarily to the high level of illiteracy among the people of this area. In addition, 
religious practise in this area tend to militate against orthodox medical practice. Community 
involvement is one of the strategies of implementing PHC effectively. This study has, 
therefore, indicated the need to create a better awareness among the people of the local 
government areas, in order to achieve a sustained high level of community participation. 

Transport is not controlled by the Medical School; the local government area health 
department is responsible for providing the students with transport from their residence to 
the health care facilities which, in some cases, are located far away on the outskirts of the 
villages. Students also rotate from one district to another, and in some cases these districts 
are located at extreme ends of the local government area. In some areas accessibility is a 
problem because some of the roads are not readily negotiable. Lack of sufficient numbers 
of 4-wheel drive vehicles to convey students and other health workers to these villages 
make effective health care difficult. 

Conclusion 

Overall, the results of this study show that the eight week community-based rural posting 
has improved the skills of medical students in the practice of community medicine. It has 
also changed their attitude favourably towards the specialty. However, there is a need for 
an effective way of improving the level of community involvement 
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Options Response (%) 

Decided before exposure to the rural posting 94 (54.7%) 

Changed decision as the result of the rural 29 (16.6%) 
posting* 

Changed to community medicine(7 from IS (51. 8%) 
Surgery, 3 from Obstetrics and Gynaecology, 

1 from Pathology, 2 from Paediatrics, 

2 from Medicine) 

Decided during or after the posting 1 1 (13.6%) 

Decided to specialise in community medicine 5 (44.4%) 
as a result of the posting (first specialty decision) 



n 

175 
175 

29 



81 
11 



* Direction of change: 

7 from Surgery to Community Medicine 

3 from O & G to Community Medicine 

2 from Medicine to Community Medicine 
2 from Paediatrics to Community Medicine 
1 from Pathology to Community Medicine 

4 from Surgery to Paediatrics 
1 from Surgery to Medicine 

5 changed their decision but did not specify the specialty. 
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Appendix 

Condensed Content of the Questionnaire on Perceptions and Career Choices. 

1. Name : 

2. Age: 

3. Sex : 

4. (a) Do you think that the period of eight weeks is adequate? 
(b) If No, how long do you think is adequate? 

5. (a) Do you think a particular rural area should be used, at all time? 

(b) If No, how many areas do you suggest, and for how long in each area? 

6. (a) Do you feel that formal lectures should be continued during the rural posting? 
(b) Give reasons: 

7. How much knowledge of the following specialties were you exposed to during the rural 
posting. 

8. How much of your expectations were met after the completion of the posting? 

1 Less than 40% 

2 Greater than 40% but less the 80% 

3 Greater than 80% 

9. In which of the following areas did you encounter problems during the rural posting? 
(please rank according to their degree of problems, i.e. the most problematic as 1 and 
the next as 2 and so on). 

Adequate facilities 
Feeding 
Transportation 
Reference materials 
Attitudes of the people 

Attitudes of the health workers and staff of the various dispensaries 
Attitudes of government officials. 

10. Has the rural posting changed your attitude towards Community Medicine in a 
favourable way? 

1 1 . (a) Did you make any decision on any specialty before the rural posting? 
(b) If yes, what specialty? 

12. (a) Has the posting changed your decision? 
(b) If yes, what is your new decision. 
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Community-Based Medical Education: The University of Maiduguri 
Experience 

B. A. Omotara, M.K.O. Padonu, A.P. Bassi and M.O. Amodu 
University of Maiduguri 
Maiduguri, Nigeria 

Abstract 

This paper discusses the Community-based Medical Education Programme of the University 
of Maiduguri since the inception of the programme in 1986. The programme has gained 
steady support from both the University and the three Local Government Areas involved. 
Some oftheprobLms encountered include lack of participation from other medical school 
departments because of paucity of junior staff and consultants, also perhaps because the 
programme is based and coordinated in the Department of Community Medicine. Efforts 
are being made to secure input from these other departments as the staffing situation 
continues to improve. Changes in ine iugistics of making the programme Faculty-based 
are also underway. 

Community participation remains a problem, although the programme enjoys a steady 
increase in the amount of community involvement. Efforts are being made to improve this 
further. 

The programme has increased the motivation of students to want to work in rural areas 
after graduation. This is considered an important factor, as it is the main objective of the 
programme. 

Introduction 

The University of Maiduguri is located in the North Eastern part of Nigeria and was 
established in 1975 as part of the third generation of Universities. The College of Medical 
Sciences (CMS) was established three years later in 1978 to train doctors who could cater 
for the health care needs of the people of the area. The initial curriculum of the medical 
school was fashioned along the lines of those of Ahmadu Bello University (ABU) which 
is the nearest university with a Faculty of Medicine. The fact that most of the founding 
lecturers/consultants were from that institution also favoured the adoption of that curri- 
culum. 

Although community-based medical education (CBME) was to be part of the curriculum, 
this was not fully implemented until 1986, eight years after the establishment of the 
Medical School . The reasons were numerous, but most notably was the lack of infrastructure, 
personnel and facilities. 

Requests for reprints should be sent to B. A. Omotara, University of Maiduguri, College of Medical Sciences, 
Department of Community Medicine, PMB 1069, Maiduguri, Borno State, Nigeria. 
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Objective of CBME 

The overall objective of CBME is to produce doctors of high quality in sufficient numbers 
to meet the health needs of the nation at community and hospital levels. The aim is to 
produce community-oriented doctors who are able and willing to serve their communities 
and deal effectively with health problems at primary, secondary and tertiary levels 
(Hamad, 1991). This objective is achieved by: 1) designing teaching and assessment to 
portray man as an integral part of his total environment; and # 2) demonstrating to the 
students, through their involvement in community medicine practice, that adaptation to the 
environment can be carried out to benefit the health of the community. At the end of the 
medical training, the student is expected to demonstrate knowledge of the appropriate 
measures (promotive, preventive and curative) that can be used to identify and solve the 
health problems of individuals, families and communities. In addition, the students should 
be able to assess the health of a community and to plan, manage and evaluate health 
services for communities (Padonu, 1987). 

Structure of CBME 

The College of Medical Sciences, University of Maiduguri, organizes a total of 14 weeks 
of community-based medical posting which is split into two: six weeks and eight weeks, 
referred to as the junior and senior community medicine posting respectively. The junior 
posting in community medicine is a block posting where all the students are posted to the 
Department for a period of six weeks. This posting is usually held after intermediate 
medicine and surgery, and junior obstetrics and gynaecology and paediatrics. It is usually 
held in the middle of the second clinical year, just about one year before the senior posting. 
This junior posting is designed to develop the data collecting skill as well as to improve 
the students 9 ability to recognise the inherent problems associated with data from 
secondary sources, so as to appreciate the need for proper data recording. 
It is a non-residential posting involving mainly urban activities. It also affords them the 
opportunity to make community diagnoses. 

During this junior posting the students are engaged in data collection, collation and 
analysis, visits to important health related establishments, a school health programme and 
any other relevant exposure commensurate with their degree of training. 
The visits to these selected establi shments are included in the posting to afford the students 
the opportunity of appreciating the problems of various establishments involved in health 
policy formulation and implementation. Table I presents the various activities of the 
students during the six week CBME block posting. At the end of the posting students are 
graded on individual reports of the visits and on group reports of the data collection and 
analysts. The marks for the posting constitute part of the continuous assessment. 

The eight week community-based rural posting is residential and is undertaken during the 
middle of the final year of the students' undergraduate education. The students will have 
completed their studies in obstetrics and gynaecology, paediatrics, laboratory medicine, 
and intermediate medicine and surgery. They will, therefore, have sufficient clinical 
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Table L Activities during the first 6 week block posting of CBME 



1986 Anthropometric measurement of nursery and pre-nursery children to monitor 
growth pattern. 

Visits to Water Board, Sanitation Board, Abattoir, Central Sewage Plant and 
Market 

1987 Anthropometric measurement of primary school children of University workers at 
Shehu Garba, mostly children of top income and civil servants, and at Gamboru, 
mostly children of low income earners. Collation and analysis of data. 

Visits as in 1986. 

1988 Blood pressure measurement of workers of Nigerian Electrical Power Authority 
(NEPA). Collation, analysis and interpretation of the data. 

Visits as in 1986* 

1989 Obtaining information from a secondary source; collection of morbidity and 
mortality records of childhood diseases from paediatric unit of the state the 
specialist hospital, collection, analysis and interpretation. 

Visits as in 1986. 

1990 Prevalence of dispepsia in both urban and rural areas, with interviews and collection 
of blood samples. Results collected, analysed and interpreted. 

Visits as in 1986. 

1991 Prevalence of asthma, based on the use of questionnaire and interview. Results 
collected, analysed and interpreted. 

Visits as in 1986. 



training to be able to provide a service to the community. The purpose of this posting is two 
fold: to expose the students to health problems in rural areas; give them experience of the 
nature and extent of community-based medical practice; and to develop further their ability 
to carry out a community-based research project, usually geared towards establishing base 
line data for a community. 

Preparation for CBME 

Preparation for the programme usually begins in August when the students return from a 
four week vacation. During this period the principles of group dynamics are introduced to 
the students. They will then organize themselves into a designated number of groups. For 
the 35-45 students a group will consist of 6 to 8 students, and the total number of groups 
for a particular year may vary between six and ten. Each group elects its leader and is asked 
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to submit two research topics in order of preference. If there is no conflict with other 
groups, the first choice of a research topic is usually assigned. In cases where a group has 
difficulty in making a choice, staff members will provide assistance, but ensure that the 
students will work on a research topic of their own interest, within the context of the 
posting. 

Prior to the students* arrival at the local government area (LGA), meetings are held with 
LG A authorities, both governmental and traditional. The chairman of the local government 
area is the chief executive of the local government He is first contacted and he will then 
direct the information to the necessary health authority and community leaders. In this way 
full community participation is anticipated. The LGAs provide accommodation and 
transport for the students for the period of eight weeks and also give them a catering 
allowance. 

Activities of the Students 

On arrival students are taken to their residence which is usually among the people or, on 
a few occasions, to a government guest house. Each LGA has three districts and each of 
the students is expected to have rotated through all three districts by the end of the eight 
weeks. 

In the morning between the hours of 8- 1 2 noon the students run clinics at the health centres, 
primary health care centres, general hospital or dispensary, depending on the available 
health care facility in a district. In the evening they interact with people in the community. 
This is when they collect data for their research project. They may also do so during clinic 
hours, depending on the nature of the research topic. This affords them the opportunity to 
get to know the people in terms of their health problems. In some instances it also helps 
them to understand the peoples' social problems, so that they can offer them useful advise. 
This is when most of the health education is carried out. For instance, the home setting 
provides the forum for advising mothers on a proper weaning diet for their babies, basic 
personal hygiene and sanitation practices. 

The service activities of the students are supervised by a medical officer who is resident 
in the health facility. Where no medical officer is attached to the facility, a staff member 
from the Department of Community Medicine will supervise the students. He commutes 
daily from Maiduguri. The project supervisor and the head of Department also visit the 
areas three times a week. The staff member from the Department of Community Medicine 
or the local medical officer receives feedback from the students and passes them on to the 
Head of Department. In this way any problem encountered in the field is reported and will 
be investigated with a view to resolving it. The project supervisor takes care of those 
problems which cannot be handled at the local level. 

At the end of the eight weeks a thesis is written, presented and defended, both by the group 
and by individual students. The presentation is made in the presence of junior medical 
students and members of staff. This is usually done to allow for criticism, so that the final 
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report, which is graded, will be of a good standard. A copy of each project is made available 
to the chairman of the respective local government, and particular reference is made to the 
section on recommendations. When each group defends its project as a group, the project 
is examined using guidelines on scientific reporting. Each section of the project is scored 
separately. For example, the introduction carries 5 marks, literature review 5 marks, 
methodology 10 marks, observation on, and discussion of results IS marks, and summary 
and conclusion 5 marks, making a total of 40 marks. Each individual defends the project 
on his/her own stating his/her major area of contribution. Each student is examined 
especially on this contribution and on overall competence in the project. The panel of 
examiners consists of the Head of Department, the project supervisor and the staff member 
or local medical officer. Each student is scored out of 20 marks, making 60 marks for the 
project. 

Problems and Prospects 

CBME is a relatively new concept in medical education, and, as it continues to evolve, 
many of the inherent problems are being resolved. Much controversy still surrounds the 
issue of whether it should be Faculty-based or department-based. This issue was debated 
at length at the 7th Biennial General Network Meeting and Scientific Conference at 
Ilorin,1991. It seems that the programme will enjoy more cooperation from other 
departments if it were Faculty-based. At the University of Maiduguri the programme is 
1 ased in the Department of Community Medicine, and it has been very difficult to secure 
the cooperation needed from other departments to make the programme more successful. 
This is partly because of the non-receptive attitude accorded to the programme, and 
perhaps because of the paucity in the number of junior staff and consultants in most of the 
other departments. 

A recent study on the attitude of final year students to the eight week rural posting* 
identified tliat the most problematic area for the students was the attitude of the community. 
Despite die efforts made prior to the students* arrival in the community and despite the 
continuity of the programme (the same LGA for 3 consecutive years), the attitude of people 
towards this programme continued to leave a lot to be desired. This is contrary to reports 
on similar programmes in other parts of Nigeria (Alausa et al., 1991). The slow pace of 
acceptance of this programme may be due to the purdah system, where married women are 
not allowed access to the opposite sex other than those in the household. If there is no 
female student in the group, access to female patients becomes impossible. Also, when 
there is a need to take blood, urine or other samples, the people usually become hostile. 
Many attempts have been made to overcome this problem. The need to properly introduce 
the students to the community has been well documented (Komolafe, 1991)**. During the 

* See the paper by Omotara, Padonu, Bassi and Amodu in this Volume of the Annals. 

** See also the paper by Omonisi, Alausa, Bunin and Bichi in this Volume of the Annals. 
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introductory period tbc traditional community leader is usually persuaded to give a sample 
of blood and urine to convince the test of his people of the lsannlessness of deprocedure. 
The use of incentives in terms of tree medication has also proved very useful in gaining 
the confidence and cooperation of the people. 
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The Nigerian Community Health Officers (CHOS) and the Service 
Needs for Primary Health Care 

M.C Asuzu and A.P. Bassi 
University College Hospital 
Ibadan, Nigeria 

Abstract 

The inadequacy of the previous western health professionals, in numbers but more 
especially in orientation and distribution, to meet the promotive, preventive and curative 
needs prompted governments to set up newer cadres of health workers for Primary Health 
Care (PHC), particularly in the developing countries. 

A questionnaire on knowledge, attitude and intendedorexpectedpractice was administrated 
to trainee community health officers (CHO's) in three institutions in Nigeria, to cover 
possible variations in the CHO students, during the 1989/1990 academic year. 98% (84 
of 86) of the ChO's responded to the questionnaire. Prior to their selection for training, 
39% of the CHO's were working in rural areas. However, only 49% were expecting to 
return to rural areas after their training. Although 90% were ready to work in the rural 
areas at least for a while, there was little enthusiasm unless living and service conditions 
were improved All trainee CHO 's saw important obstacles to PHC in Nigeria which were 
considered to be primarily of a managerial nature. 76% expressed a desire for more 
surgical training beyond that provided in the present course. 
The study shows that the same managerial problems that prevent doctors from serving in 
rural areas apply to these newer cadres as well More attention should, therefore, be paid 
to these problems in the interest of optimal deployment of the CHO's. 
SimilarattentionshouUbepaidtotheseprobUmswithaviewtoat^ 
reasonable involvement in PHC at the grass roots. 



Introduction 

Primary Health Care (PHC) - the key to the attainment of Health for all by the Year 2000 
- had been defined at the Alma Ata Conference as essential healthcare based on practical, 
scientifically sound and socially acceptable methods and technology, made universally 
accessible to individuals and families in the community through their full participation or 
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involvement, at an affordable cost and in the spirit of self-reliance(World Health 
Organization, 1978). The training of new cadres of health and health-related workers for 
PHC has been undertaken by many developing countries in pursuit of their PHC 
programmes (Lambo, 1990; Oosterberg, 1988; Edghill, 1980), The most important 
justification for this emphasis has been the failure to orientate, reorientate, and/or produce 
an even and adequate distribution of health workers within the countries' health services, 
especially doctors. Without any observable effort to find out the reasons for such 
maldistribution, or, if already known (Morgan et al., 1973), without seriously attend to 
them, many governments have zealously pursued the training of newer cadres of health 
workers. 

The present study was, therefore, carried out to establish the extent to which the 
professional background and anticipated practice of trainee Community Health Officers 
was likely to succeed where western trained health workers did not meet the needs of even 
distributional and appropriate orientations towards PHC. Three educational institutions 
in Nigeria which trained CHO's during the 1989/1990 academic year (one Oniversiy 
teaching hospital-based programme, each at Maiduguri and Ibadan, and one at a school of 
hygiene/health technology in Ibadan) were used in the study to cover all the possible 
variations in the background of the trainees. 

The CHO Training Programme 

The CHO's are trained to work as alternatives auxiliaries to general medical practitioners 
or community physicians, especially in those areas where such doctors are not available. 
CHO's are thus expected to practise at comprehensive and primary health care centres in 
urban and especially in rural areas. Initially, they were to be recruited from among 
experienced nurses/midwives, and public health superintendents, as well as from rural 
health assistants and superintendents. The duration of their training was to be one calendar 
year, and the curriculum covered all aspects of community and basic medical care at the 
district level of health care in the country. Even though all the training centres were located 
in the big cities, the candidates were required to undertake a residential community and 
primary medical care practicum in a rural setting lasting up to three months. There were 
12 such training centres in the country at the time of this study. Competence at the end of 
the course was assessed internally within each school and externally by national assessors. 

There has been some uncertainty as to the career structure and prospect for these CHO's 
within the local, state and national (federal) health services. For most of the CHO's, their 
training and certification has only served as an extra qualification to help them with 
promotion within their original professional career as nurses/midwives or as environmen- 
tal health officers. 

The qualification also helped when applying for the post of PHC co-ordinator at local or 
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state government level, where there were no medical officers of health (Community 
physicians) as coordinators. Recruitment for the course was by personal application. 
However, because of the cost of the course, agency or institution sponsored candidates 
were more likely to apply, be selected to take up their place and complete the course. 
Although there were a few health centres which were used almost as models for PHC and 
thus acquired more equipment for appropriate practice of PHC, generally no effective 
effort was made to equip PHC centres better. There were also no extra incentives to make 
the posts where the CHO's are expected to work more attractive. 

Materials and Methods 

In this questionnaire-based study, information was collected on the location and profes- 
sional background of the candidates before they came for this training. Their knowledge, 
attitude, expected location of practice and other details relevant to PHC were also 
collected. 

In order to encourage honesty in answering the questions the CHO candidates were assured 
of individual anonymity in the survey. They were asked to give their identity number only 
if they were willing to do so. Identity numbers would allow us to identify students, if need 
be, for any clarifications of their answers. 

Results 

Eighty-four of the eighty-six (98%) CHO's in training in the three institutions answered 
the questionnaire, 28, 33 and 23 from the three centres respectively. Table I shows the job 
designation of the respondents before their admission for training. The breakdown differs 
from one institution to the other, with more non-nurses at the Maiduguri centre. The job 
location of the candidates before, and the expected location after training, are shown in 
Tables II and III. Even though the selection of rural rather than urban location improved 
over all in the other two centres, it deteriorated at the UCH centre. Although the majority 
of the CHO trainees were selected from urban centres, a good proportion were ready to 
serve in a rural area (90%). However, most of them preferred to work for only a short period 
in such areas, unless the numerous obstacles for doing so were removed or meaningful 
financial/economic incentives were provided (Table IV). 

The scope of the CHO training was felt to be inadequate by many of the candidates, 
especially in relation to clinical skills (Table V). Only 2 1 % were satisfied with the training 
programme. Though the emphasis was on promotive and preventive health care, and many 
approved of that emphasis (Table VI), many of the 79% who did not feel convinced of the 
adequacy of the course coverage, listed additional items and indicated a desire for more 
surgical skills to be included in the training. 
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Table I. The Job designation of candidates before CHO training 



Job Description 


U.CHIb. 


S.O.H. lb. 


UMTH.M 


Total (%) 


Nurses 


5 


33 


7 


45 (53.6) 


Public Health Nurses 


15 






15 (17.8) 


Community Health 


8 




16 


24 (28.6) 


Total 


28 


33 


23 


84(100) 


Table JJ. Job location before training 


Location 


U.C.H. lb. 


S.O.R lb. 


UMTH.M 


Total (%) 


Rural 


12 


19 


12 


33 (39) 


Urban 


16 


14 


21 


51 (61) 


Semi-Urban 










Total 


28 


33 


23 


84(100) 


Table m. Intended or expected job location after training 


Location 


U.C.H lb. 


S.O.H lb. 


UMTEM 


Total (%) 


Rural 


7 


21 


13 


41 (48.8) 


Urban 


4 


5 


7 


16 (19.0) 


Semi-Urban 


17 


7 


3 


27 (32.2) 


Total 


28 


33 


23 


84(100) 
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Table IV. Volunteered length of stay in rural areas after training 



Period 


U.CJBL lb. 


S.O.H. lb. 


UMTH.M 


Total (%) 


3 Years 
Life long f 


25 

3 


30 
3 


21 
2 


76 (90.4) 
8 (9.6) 


Total 


28 


33 


23 


84(100) 


Table V. Attitude of students to the carreer structure for CHOs 


Adequacy 


U.C.H.Ib. 


S.O.H. lb. 


UMTH.M 


Total (%) 


Yes 


6 


7 


5 


18 (21.4) 


No 


18 


21 


15 


54 (64.3) 


Not sure/ 
do not know 


4 


5 


3 


12 (14.3) 


Total 


28 


33 


23 


84(100) 


Table VI. Perception of the trainees of the preventive and promotive emphasis in the 
training programme 


Perception 


U.C.H. lb. 


S.O.H. lb. 


UMTH.M 


Total (%) 


Very Important 


24 


28 


15 


67 (79.8) 


Minimally important 


4 


5 


6 


15 (17.8) 


Not so important 






2 


2 (2.4) 


Total 


28 


33 


23 


84(100) 



All the CHO trainees agreed on the existence of obstacles to the implementation of PHC 
and the achievement of health for all in Nigeria (Table VII). The problems listed by the 
trainees were categorised into three broad areas as shown in Table VII. Poor remuneration 
for community or primary health care workers, inadequate career structure for the CHO' s 
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and very poor living and service conditions for rurally located health workers featured 
prominently in the responses. 



Table VEL Obstacles to the success of Primary Health Care in Nigeria 



Obstacles 


U.C.H. lb. 


S.O.H. lb. 


UMTH.M 


Total (%) 


Managerial 


18 


24 


17 


59 (70.2) 


Lack of political will 


* 

5 


6 


1 


12 (14.3) 


Implementation 


5 


3 


5 


13 (15.5) 


Total 


28 


33 


23 


84(100) 



Discussion 

The major justification for the training of newer cadres of health workers for PHC is the 
inadequacy of existing health workers in numbers, distribution and especially in their 
orientation to meeting the needs of complete community health care for all the people. This 
applies especially to the new grades of primary health workers in the range of nurses to 
doctors, including the Nigerian CHO's. The yardsticks for assessing the success of the 
CHO programme should, therefore, be: 1 ) adequacy in numbers for the various communities 
in the country; 2) actual distribution to these communities; 3) adequacy of the training 
programme, or the candidates themselves in being properly orientated to community 
health care rather than an undue immersion in mere curative work. 
Those aspects of these measures that can be evaluated by a study of the CHO trainees 
themselves were incorporated into this enquiry. At the Maiduguri centre candidates were 
selected from Northern Nigeria. Because of the greater lack of western trained health 
workers in this part of the country, they included a large number of the lower cadres of the 
newer and older PHC workers, the community health supervisors and rural health 
assistants/superintendents. The Ibadan Teaching Hospital candidates were selected from 
all over the country and reflected the spread of all types of the established western health 
workers in the range between nursing and medicine. The School of Hygiene, Ibadan 
CHO's were exclusively senior nurses from all over the South Western States of Nigeria 
undertaking a sandwich Public Health Nursing and CHO course. 
The selection of the candidates (Table II) does not reflect the composition of the 
community of the country which is made up of some 75-80% of rural dwellers. The 
anticipated places of relocation after CHO training (Table III) did not reflect this 
distribution, except in the urban to semi-urban shift. This shift is in the right direction and 
may be a pointer to the fact that CHO's might perhaps be prepared & meet the needs of 
professional distribution for PHC, if at least some of the major obstacles were removed. 



53 



For this reason it is important to pay particular attention to the remediation of the obstacles 
to ensure adequate remuneration of PHC staff to reflect the claims being made for its 
importance. Enticing career structures for PHC staff, and the development of appropriate 
infrastructures and amenities - piped water, tarred roads, good schools and reliable 
electricity would represent essential requirements. 

In relation to career structure, it would be interesting to consider the advantage? of 
permitting CHO's who perform well to train further to become community medical 
officers, as is the case in the PHC programme in the Philippines (Oosterberg, 1988). That 
programme is run by the University of the Philippines, sponsored by the Government but 
independent of the medical school. This programme is perhaps the very thing the CHO's 
mean when they asked for more surgical training. However, it is more likely to be an 
indication of the inadequacy of the orientation of these candidates, even at this time, to the 
preventiveandpromotiveaspectsofPHC,raiher thanes 

of the CHO' s to stay in the rural areas for long, unless a lot of the obstacles were corrected 
(Table IV) reflects the same reasons why doctors are reluctant to embrace community 
Medicine or indeed to practice PHC as currently provided (Edghiil, 1980; Morgan et al., 
1973). It is, therefore, possible that, if these obstacles were corrected, we could even expect 
to see doctors opting for Community Health and PHC practice at the district (Local 
Government), level, judged by the World Health Organization to be the operational level 
of PHC. We ought to be able to provide one such medical officer of health for each of 
Nigeria's Local Government Areas as the minimal norm. Without such meaningful 
involvement in PHC (i.e. including doctors, and adequate remuneration, it would appear 
that both community health, as well as the more precisely defined PHC dimension in our 
health services will continue to be taken as a second class type of health care. PHC would 
then fail to achieve the hopes that have been placed on it, or progress too slowly and at a 
very much less cost-effective rate. 

A large, nationwide (total) study of trained CHGs is needed to assess the programme. We 
expect to undertake this study. 
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Ke Experience in Community-Based Education for Health Professio- 
nals 

Musa K. Jinadu and A. Davies- Adetugbo 
Obafemi Awolowo University 
Ile-Ife, Nigeria 

Abstract 

TheFacultyofHealthSciencesofObafemiAwohwoUniversitystanedwith 
community-based and primary health care-focused curriculum in 1972. This paper 
describes the initial educational programmes of the Faculty and focuses on the planning, 
implementation and evaluation of the community-based aspect of the curriculum. This was 
subsequently neglected as new departments were createdandnew staff were recruited into 
leadership positions in the Faculty. In 1987, a revised community-based educational 
programme was developed. Adoption of primary health care (PHC) as a national health 
policy provided the main impetus for the change. Institutional mechanisms for planning, 
coordinating and evaluating the programme are described. The paper concludes with 
lessons learnt from the programmes. 

Ife Philosophy, Objectives and Educational Programmes 
The Faculty of Health Sciences of Obafemi Awolowo University, Ile-Ife, was established 
in 1970 by the former Western Government of Nigeria. Its foundation students were 
admitted into the Faculty in 1972. 

Before the inception of the Faculty, there were variously Circulated concerns for the 
relevance of medical and other health professional education to the health needs of the 
nation. Ife was the first university in the country to translate these concerns into reality, as 
shown by its educational objectives and service programmes (University of Ife, 1981- 
1983). 

These included a clear commitment to: a) train a health team of graduate and non-graduate 
medical, dental, nursing and other personnel for the provision of comprehensive health 
care in any community in the State and the nation; b) train health personnel with 
considerable knowledge and skills to work effectively in both urban and rural areas of the 
nation; c) provide comprehensive health care to a defined geographical area of the State; 
and d) involve members of the community in the provision of health care to their 
communities. 

Requests for reprints should be sent to M.K. Jinadu, Obafemi Awolowc University, Faculty of Health Sciences, 
Department of Community Health, Ik-Ife, Nigeria. 
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The Faculty provides undergraduate programmes in medicine (MB, CHB), Dentistry 
(B.Ch.D.)t Nursing (B.N.Sc), Medical rehabilitation (B.M.R.) and Environmental Health 
(B.Sc. EVH). There are also post-graduate programmes in various health and medical 
sciences and clinical professional disciplines of the Faculty. 

The initial curriculum made a four year B.Sc. Health Sciences Degree a prerequisite for 
training in the clinical aspects of medicine and dentistry. The revised curriculum of 1972 
removed the compulsory B .Sc. Degree, reduced the length of medical and dental education 
to six years and introduced intercalated B.Sc. Medical Science, "to a well-selected group 
i>f promising students who would become future teachers in the basic medical sciences in 
which t^ere is an acute national shortage of Nigerian experts in all the existing medical 
schools" (University of Ife, 1989-1992). Table I shows current undergraduate degree 
courses offered by the 
Faculty and the duration of the courses. 



Table I. Duration of Ife educational programmes for health professionals 



Medicine Dentistry 

(MB.Ch.B) (B.ChD) 


Nursing 

(B.NSc) 


Medical 
Rehabi- 
litation 

(BMR) 


Environ- 
mental 
Health 

(B.ScEVH) 


B^c. Medical 

Science 

(intercalated) 


Preliminary 
courses: 
lyr. 


lyr. 


lyr. 


lyr. 


lyr. 


lyr. 


Preclinical 
courses: 
2yrs 


2 yrs 


lyr. 


lyr. 


lyr. 


(after precl.) 


Clinical 
courses: 
3 yrs 


3 yrs 


3yr. 


2 yrs 


4 yrs 





Between 1 972- 1 982, medical and dental students were admitted to a four year B .Sc. Health 
Sciences course which preceded the 3 year clinical training. 



Since the inception of the Faculty, teaching and learning activities involve the use of 
learning objectives, seminars, group discussions, self instructional techniques, and integrated 
lectures. Considerable emphasis was laid on training the different health professionals 
side-by-side. The students were exposed to similar teaching and learning activities during 
their preliminary and preclinical years. They took the same science courses - chemistry, 
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biology, physics and mathematics - during their first year of University education. These 
courses were based in the Faculty of Science and were organised for science-based degree 
programmes. During the second year, nursing and medical rehabilitation students attended 
the same courses in anatomy, physiology, biochemistry, biostatistics, and behavourial 
sciences as medical and dental students, before concentrating on their various clinical 
courses during the third and fourth years. 

Initial Community-Based Educational and Service Programmes 
The Faculty adopted from the very start a defined geographical area, known as Ife-Ijesha 
zone, for its health service programmes. The purpose was to fulfill the objectives of 
producing health personnel that could function effectively in both urban and rural areas of 
the country. 

The area of 3732 square kilometres had a population of about 1 .2 million 
inhabitants in 1972, living mainly in about 700 villages and towns. 
Four general hospitals and four PHC centres in the area were acquired by the Faculty for 
services, teaching and research. Health committees were formed for community participation 
in the PHC programmes . Members of the committees included traditional rulers, community 
leaders, senior health officers, and workers in other sectors such as education, information, 
and agriculture. Multidisciplinary health facilities location and needs assessment surveys 
were conducted to obtain baseline information for the planning of a model basic health 
service for the catchment area. 

The original community-oriented part of the medical curriculum was taught mainly during 
the last three years of clinical education. The community-oriented educational programme 
was based in the Division of Community Care, the forerunner of the Department of 
Community Health, with centralised planning and coordination which involved all 
members of the Faculty. The objectives of the programme included the ability of the 
student to: a) "assess the health status of a community, define its major health problems, 
allocate priorities and formulate plans for dealing with them; b) work in or lead a health 
team; c) undertake community health education; and d) tailor his/her activities in respect 
of the health needs of the community, so that they can reflect the social, economic, 
psychological and ecological factors of the area" (University of Ife, 1976-1978). 

The students spent about 10% of their time in Years 5 and 6 and 25% of their time in Year 
7 on the community-oriented programme. They were posted to PHC facilities in the rural 
communities to conduct community-based investigations under supervision and to render 
services in the communities and at the PHC centres. 

The Faculty provided transport and accommodation for the students and their supervisors. 
A certain amount of money was allocated by the Faculty for community involvement 
activities and a part of this was u^ed for running the programme. Other support for the 
programme came from the community through the community health committees. 
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A major drawback of the programme was that the community-based components of the 
educational activities of the students were not adequately evaluated for their effectiveness 
and efficiency. However, experience showed that the programme was enthusiastically 
accepted by students, staff and the community. By 1982, when the programme was no 
longer implemented regularly, one community sent a delegation to the Department of 
Community Health to find out why the 'student doctors* were no longer seen in their 
community. 

Problems Encountered with the Original Community-Oriented Curricu- 
lum 

By the end of the 1970s the community part of the original curriculum began to suffer from 
benign neglect New departments were created from the Division of Community Care, and 
new staff, who were not committed or orientated to the original philosophy and objectives 
of the Faculty, were recruited into leadership positions. The centralised approach to 
planning and allocation of resources to the community-based educational and service 
programme gave way to decentralised, departmental programmes. Each department began 
to concentrate its resources into the teaching of its core areas of specialisation. Resources 
for supporting and maintaining the community-oriented programme, therefore, dwindled. 
Those of the Faculty members who were not committed to the original philosophy and 
objectives of the Faculty pressed for a change of the programmes in favour of the traditional 
approach. Although the revised curriculum for medical education in 1982 stressed 
"commurity-oriented medical practice" ami, "early intixxluctiontocommunitymedicine M , 
the students* exposure to community health programmes never went beyond the confines 
of a few health centres, mainly in urban and semi-urban areas. 

Revise Community-Based Educational Programme 
Since the middle of the seventies, PHC became a major focus for health service 
programmes in Nigeria (Federal Ministry of Health, 1976) and was perceived as an 
approach for addressing the health needs of the majority of the population of the country, 
especially those living in rural and semi-urban areas (Federal Ministry of Health, 1978). 
The National Health Policy, formulated in the eighties, was based essentially on the PHC 
philosophy, objectives, strategies and programmes (Jinadu, in press). It, therefore, became 
very obvious that, in order to make medi.il education relevant to the needs of the country, 
a substantial part of the cuiriculum should be devoted to PHC educational programmes. 
PHC cannot be taught in a vacuum; communities and their resources have been accepted 
as laboratories for medical schools in developing countries. 

Therefore, the Faculty of Health Sciences decided in 1987 to re-examin? the PHC aspects 
of its medical curriculum and to formulate a programme for its implementation. A 
committee was formed, and its members were drawn mainly from the departments of 
community health, paediatrics/child health, environmental health and epidemiology, and 
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obstetrics and gynaecology. Tins committee developed a revised community-based 
education, service and research programme called Rural Comprehensive Health Programme 
(RUCO). The programme is based in the Dean 9 s office, and its general objectives are to: 
a) provide organised rural community exposure to trainees in health sciences; b) provide 
facilities for individual and interdic ipli nary research relevant to the needs of the community; 
and c) make a measurable impact, on an on-going basis, on the health of the community. 

The students' educational objectives include: a) community diagnosis through collection, 
analysis and interpretation of community-based data; b) self-directed learning; and c) 
initiating and evaluating lines of action for solving identified community health problems. 

The RUCO programme is located in the Imesi-Ile and Isoya areas of Obokun and Ife central 
local government of Osun State. The two areas are within the Ife-Ijesha community, the 
previous catchment area of the Faculty . Imesi-Ile area consists of a group of 1 2 villages and 
towns with a total population of about 200,000 inhabitants. A rural health centre, which 
was acquired by the Faculty at its inception, has been upgraded to a modern comprehensi ve 
health centre for training, research i-id services in a rural setting. The Isoya area consists 
of 11 villages with a total population of about 6,100 inhabitants, located 15-45 kilometres 
from the University. 

Medical students are posted three times in this programme. The first posting is for four 
weeks in Year 2, the second is for four weeks in Year 4, and the third is for eight week in 
the final year. For the first posting the students are divided into small groups of 5-10, 
depending on the size of the class. The groups carry out community-based activities that 
are related to surveys of community needs. During the subsequent postings, the students 
carry out specific community-based problem solving research activities and participate in 
the delivery of PHC in the rural communities. 

Assessment of the students 9 learning activities are based on: 

- Seminar presentations of project reports; 

- Progressive written examinations and end of the course examination; 

- Attitudinal assessment of the students by their supervisors; 

- An assessment of the Community Health Project by independent assessors. 

These items are given different weighting before the scores are added together. 
A coordinator is responsible for planning and programming the educational activities, as 
well as for the maintenance of the programme . Academic and technical staff of the Faculty, 
especially those from the Department of Community Health, arc responsible for supervising 
the students in the community. Recently, a PHC committee was formed and headed by the 
Dean. Members of the committee are drawn from various departments of the Faculty. The 
committee that ensures the sustainability of the programme is responsible for evaluation. 
Although no formal evaluation of the programme has yet been conducted, experience so 
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far has indicated that the programme is potentially able to achieve its objectives. The 
students and the community have reacted favourably to the programme. Members of the 
rural communities have always welcomed the students and staff, especially where the 
students and staff have not only collected survey data, but also provided some services. 

Implementation of the RUCO programme is not without its problems. The main problem 
is lack of finance. The programme has no vehicle of its own for taking students and staff 
to and from the rural areas. It has to compete for transport with other clinical programmes 
in the Faculty; quite frequently the students are unable to conduct the community-based 
activities in the designated rural areas. Other essential equipment and supplies for the 
smooth running of the programme are often not available. This lack of equipment leads to 
frustration among the staff and students. The problem could be resolved, if it were specially 
funded by the government or by external sources. 

Conclusions 

Medical schools in developing countries that aspire to make their educational programmes 
relevant to the essential health needs of their countries can no longer afford the luxury of 
ignoring PHC education, services and research. An effective way of achieving this is to 
strengthen the community-based curriculum. However, adequate implementation of the 
community-based educational programme requires appropriate funding. Adequate 
institutionalisation of the programme and education of new staff through seminars and 
workshops are also essential. 
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Community-Based Experience and Service (COBES) Programme 

Ben A. Aiyedun 
University of Ilorin 
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Abstract 

This paper on the Ilorin community-based programme includes a general description of 
the education and service elements in the various postings of medical students during their 
undergraduate course. Emphasis is given to the careful preparation of government and 
local authorities to ensure smooth acceptance of the students in the community. An 
Appendix sets out the general and specific objectives of these educational experiences. 

Introduction 

The University of florin Medical School is one of the newest medical schools in Nigeria. 
It was started in 1976 with 46 students drawn from various parts of Nigeria. It operates a 
six-year medical programme for students following their high school education. 

At the inception of the Medical School, the funding fathers wanted to do something 
different from other conventional medical schools. They planned a curriculum that was 
integrated, student-centred and problem-solving in concept The disciplines of anatomy, 
physiology and biochemistry were taught together by three or four lecturers, each having 
an input into the body-system being discussed by the class. The programme was a 
revolution in medical education in Nigeria. 

The part of the curriculum that has received the best comments and popular acclaim by 
older medical schools in Nigeria is the Community-Based Experience and Service 
(COBES) Programme of the Faculty. We have recently been designated a Collaborating 
Centre for Health Manpower and Training of the World Health Organization. 

In this programme, the students are sent out to the community at the beginning of their 
studies to see what health knowledge they can gain and develop during this posting. The 
concept of early exposure to the community was based on the problems that faced Nigeria. 
The doctors produced by the medical schools had not been willing to work in rural 
communities. The Faculty decided that if students were orientated towards Rural and 
Community Medicine at the very beginning of their six-year programme, they would gain 
a feeling for the services needed by rural communities and might cultivate a liking for 
service in rural areas. 

Requests for reprints should be sent to Ben A. Aiyedun, University of norin, Faculty of Health Sciences, 
COBES Academic Planning Committee, PMB 1515, norin, Nigeria. 
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The aims of the COBES programme are: 

1. To train medical students in community public health in different rural settings, 

2. To let the student see and appreciate local health problems early in their medical 
training. 

3. To make the Medical School acceptable to the Community. 

4. To put medical theory into practice. 

5. To implement an interdisciplinary approach to medical education that is not limited to 
any one department 

6. To render service to the community (see COBES objectives in the Appendix). 

This programme needs a great deal of planning and funding to make it effective. The 
Faculty appoints a COBES coordinator to oversee of the successful implementation of the 
posting. Two Faculty-based committees assist the coordinator in his duties, the Academic 
Planning Committee (APQ, responsible for the academic content, and the Logistic 
Planning Committee (LPC) that arranges transport, catering, housing, medical care and 
other logistics. 

Each class of students is divided into four or more groups of 20-30 students. In order to 
make the students acceptable in the communities, female students are evenly distributed 
amongthegroups.Theywillbeabletoconto^ 

be visited by male students. Each group has four or five staff members as advisers. As far 
as possible, the students are encouraged to live in the community during their posting. In 
their second year, the students spend four weeks in the community, a further four weeks 
in their third year, and three weeks in the fifth yea- of their undergraduate education. 

Reaching the Community 

At the commencement of every posting, each group of students and staff meets together 
with the heads and the elders of the villages. The purpose of the posting is explained to the 
village head and his team, and their cooperation is sought The village head will then 
announce the presence of the student through the town crier and in churches and mosques 
on religious days, so that the community can be aware of the presence of the students. 
Each house leader is then advised to so inform every member of his household, so that the 
students will have a problem-free posting. 

COBES Postings 

The students start each posting with an exercise in demography. They have to draw a map 
of the area with the number of individual houses and identify the available health facilities. 
From then on, they can carry out different surveillance studies on the chosen topic of the 
posting. Examples of past topics are: parasitic diseases, mental illness, alcoholism, E.P.L, 
drug abuse, diarrhoea diseases, skin infections, blood pressure, ill health in school- 
children. 
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The COBES sites are carefully chosen, so that the students can move to different areas to 
do their demographic w orfc in order to avoid repeating the same exercise and to prevent 
students from copying old reports. This has made annual interviews bearable for the 
communities. The second posting usually uses the WHO theme of the year. This has 
involved house-to-house interviews with the inhabitants and physical examination of 
members of different households. In some cases where people work away from their 
homes, the students have to repeat their visit in the evening. 

Each group has a base where the staff members supervise the use of drags for minor 
ailments. Students refer to members of the staff those patients who needed urgent 
treatment Serious cases are referred to the teaching hospital. 

At the inception of the programme, the Faculty did not plan to treat patients in the 
community. As time progressed, health care became necessary, in order to ensure 
continuing support and cooperation from the residents during the two to three 4-week 
postings each year. A large chunk of the COBES budget has had to be used for the 
procurement of drugs and dressing. The addition of service to our programme has greatly 
boosted cooperation and support for the COBES programme in the area where it operates. 
The students have also been able to assist the communities with watersupply , latrines, and, 
in one instance, electricity. Such help has been possible through the contact which the 
students were able to make with authorities such as the State Government, Local 
Government and the Directorate of Roads and Rural Infrastructure (DFFRI), as well as 
with the Rotary and Lion Clubs. 

Assessment of Students and Reports 

At the end of each COBES posting, a day is set aside where members of each group present 
their findings and results of the COBES posting to the other members of the class and staff 
members of the Faculty. The presentation is assessed by Faculty staff and outsiders. The 
students also undertake a three hour written examination, and the staff supervisor of the 
groups assesses each student's performance on site. A comprehensive report is prepared 
subsequently by the COBES coordinator for the Faculty, the University authorities and 
interested parties like the Ministry of Health and the Local Government to justify 
continued funding for the programme. 

Discussion 

The University of Ilorin Medical School has taken great care to sell its programme and its 
students to the community, a marketing strategy. The programme has also sold medical 
education to the students. This has been due to the Faculty* s policy of orientating students 
towards community based problems and finding solutions for them. One of the major 
problems of Nigeria's Health Delivery Services has been the inability of Government, 
State and Federal, to induce doctors to work in rural areas. This problem was paramount 
in the mind of the designers of the COBES programme. If students can be encouraged to 
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undertake community work as they are taught in COBES, they may not object to working 
in rural areas later on in life. 

The programme has involved careful budgeting. This is necessary in order to cut costs to 
the barest minimum and yet achieve the best possible results. At an annual financing 
session, chaired by the coordinator, each group leader requests a budget by defending his 
needs for beds and bedding, catering, transport, housekeeping, and drugs and dressings. 
This should help medical students, early in their training, to appreciate the need for prudent 
spending. 

Local politics has also been involved in the COBES postings. The contact between the 
students, village and local heads and religious and local government leaders is very 
encouraging and could be strengthened. Local leaders need to be deeply involved and 
informed if appropriate arrangements for health care are to be effective. As a result of the 
Faculty's efforts some communities have formed local committees that have taken a keen 
interest in the COBES activities of the students and they have greatly assisted the students 
in organising health talks and demonstrations for the communities. 

Information on the programme has been widely disseminated by the student presentations 
and the coordinator' s COBES reports which are made available to the Ministries of Health, 
Local Governments, University, and staff and students of the Faculty. The programme has 
used the available resources of the Faculty to the immense benefit of everyone concerned. 

Conclusions 

The COBES programme has been widely publicized in the press, radio and T.V. of the 
State. The Federal Government, due to the constant requests for funds through the Ministry 
of Health, the National Universities Commission and the University, has built two 
Comprehensive Health Centres for the Medical School in the State, so that the programme 
can expand and flourish. Recently the Faculty has constructed two student hostels, so that 
the students can live on site. 

Older medical schools have praised the programme, and the graduates of the first eight 
classes are registered medical practitioners in the country's Health Delivery Services. 
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APPENDIX 

Terminal Objectives 

At the end of the course, over a period of six years, the new graduate should: 

1. Be sensitive to Community Health needs. 

2. Be able to establish a relationship with his patient and his background. 

3. Be seen to contribute to and assist Community Health effort 

4. Be able to work in any community on his own with adequate knowledge of the influence 
of culture and socio-economic situations on his patients. 

5. Have team spirit towards solutions of health problems of the community. 

6. Have developed habits of individual study. 

7. Be able to organise a health scheme for a school or a community. 

Limited Objectives >• 
A. Surveys 

A.l Plan and effect a simple estimate of the population in a given area. 
A.2 Use the data obtained (in A.l) and collect facts by means of surveys on the health 
needs of the area. 

A.3 Analyse the data (in A.2) to design a plan of priorities for action to improve the health 
of the community. 

A.4 Carry out community-wide measu* *s or campaigns (designed in A.3) with the help 

of relevant members of the team. 
A.S Monitor the effects of health measures or campaigns and modify these as necessary. 

A. 6 Communicate findings of his work and relate them to the development of medical 

care and scientific advances. 

B. Clinics 

B.l Supervise the workers of a health team and lead them. 

B .2 Run clinics and perform home visits for patients with social problems and report on 
two of them. 

B. 3 Maintain necessary supplies for a clinic and work with the relevant health authority 

to achieve this. 

C. Environment and Culture 

C. 1 Gather information about prevailing attitudes to health and disease in any community 
and use it when planning the health care of individuals or of the community. 

C.2 Identify climate and seasonal factors which affect the patterns of disease, nutrition, 
work and movement of people in any area and use such data in health planning. 

C. 3 Identify aspects of culture that are injurious to health and learn to counsel on them. 

D. Work 

D. 1 Given a workman with disease, identify the effect of the course and outcome of his 
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disease on his work, and the effects of his work on his disease. Act to enable him to 

woik effectively* 
D.2 Rehabilitation of a disabled workman in the community. 
D.3 Be able to advise on health measures in a factory, in a workshop. 

To enable him to achieve these objectives, the student should be equipped: 

a) to find out as much as possible about the community in which he is working; 

b) to analyse the information he collects; and thus c)to apply it wherever it affects 
decisions about health and disease. 

Attitudes, Customs, Habits 
Thus he should be able to: 

1 . Communicate with members of the community and seek to be a member of it rather than 
to continue as a stranger within it. 

2. Gain the confidence of its leaders and of its people through disciplined service. 

3. Record the prevalent believes about the cause, practices related to the prevention and 
cure, and fears about the effects of common diseases. 

4. Identify (from 3. above) those believes, practices, fears and attitudes which are 
detrimental to health; plan means to correct them, and put these plans into practice 
through members of the community and of die health team. 

5 . Record the common customs of the community related to pregnancy, childbirth and the 
puerperium and identify those which may adversely affect mother or baby, and by 
planned education of mothers through the health team seek means to change such 
customs. 

6. Identify the hierarchical structure of the community, its leaders, and those to whom 
preference, privileges or authority is given, and use the knowledge of this structure in 
all plans related to health and its promotion. 

7. Identify and, wherever possible, record and measure those influences/factors within the 
community which are changing its traditional practices and values, and take account of 
such influences/factors in all plans for its health services. 

8. Given the fact of change, identify the prevalence of those habits relating to alcohol, 
tobacco, drugs, self-medication, methods of transport and the abuse of sex which may 
affect health, and take all means possible to inform the community about their known 
dangers. 

Nutrition 

1 . Identify the staple foods of the community, record methods of production and storage 
and supply, together with the cost and availability of supplementary foods, at all 
seasons of the year. 

2. From 1., identify the seasons of the year when supplies are short and the specific 
nutritional deficiencies which are liable to arise. 

3. From 1. and 2., identify those people or groups of people within the community who 
are particularly at risk of nutritional deficiency and devise a plan to ensure that they are 
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protected. 

4. Record the practices used in weaning and feeding infants, and in feeding other members 
of the family, and identify the effects of these practices on the nutritional value of the 
food and on the contamination of food with micro-organisms. 

5. Identify practices which art detrimental to health and plan how these can be changed 
through education and other necessary means, and take initial steps to Implement these 
plans. 

6. Record the changes in eating habits in the community, identify the likely effects these 
will have on health, and cooperate with community leaders through the health team in 
education about the effects of these changes. 




Medical students trying a bore-hole pump constructed with 
their assistance for a COBES site village. 
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Abstract 

The foundation members of the undergraduate programme for the 5 year medical training 
programme began during the 1986/87 academic session in Bayero University Kano, 
Nigeria. The medical school from its inception, adopted an innovative medical education 
philosophy with a programme of relevance to community health needs - called Community* 
Based Medical Education and Services Programme (CBME & SP), a programme which 
emphasized early introduction of students to the community for learning and service. The 
programme, therefore, included field postings. 

Preparations for the implementation of the programme took approximately 10weeks,from 
January to March, 1987. Formation of a planning committee by the Faculty was seen as 
a focus for staff involvement The main responsibility of the committee was to develop 
strategies for the CBME & SP. Tlit staff involved in the programme were introduced to the 
ideology during seminars and a workshop. 

Permission was obtained from relevant government agencies to allow us to approach 
communities and their health services for the purpose of using them for our field 
operations. 

Three members of the Planning Committee visited 21 communities and sought their 
cooperation. The visits enabled the team to take oral evidence of what the people regarded 
as their health problems. Using a 76 point rating scale, two communities were selected 
The objectives and curriculum contents were then drawn up. 

Several follow-up visits led to free accommodation for the students and staff during the 
field posting. The District Heads requested the village heads (Dagachi) to ensure full 
cooperation by the ward heads. 

Requests for reprints should be sent to M. Kola Omonisi, Department of Epidemiology and Community 
Health, Faculty of Health Sciences, University of Ilorin, PMB 1515, Ilorin, Nigeria. 
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The students had a number of preparatory lectures in the orthodox basic subjects and they 
were taught how to prepare survey instruments and how to conduct interviews. 
Finally, the Faculty's Minimum Learning Objectives were given to the stodcnts, and a joint 
meeting of the staff and students discussed the roles of the Faculty staff. Lack of sufficient 
reading material, lack of staff experienced in community-based work, and lack of transport 
proved to be serious constraints. 

Introduction 

The acknowledgement of the values of community-based/oriented programmes as cardinal 
to the training of health professionals motivated the Dean of Bayero Medical School to 
entrench in the medical curriculum a programme entitled Community-Based Medical 
Education and Services Programme (CBME & SP). The Bayero Faculty of Medicine 
accepted the initial intake of 22 students (16males and 6 females) for the 1986/87 academic 
session, some ten years after the establishment of the Bayero University. 

Aims and Objectives of the CBME & SProgramme 

The Community-Based Medical Education Programme is essentially a stage-by-stage 
field training programme. The purpose of initiating this field training programme includes 
that the students should learn: 

- how to approach a community in an acceptable manner, either as groups or individuals; 

- how to communicate effectively with the community; 

- how to work with the people of a defined community; 

- how to plan and execute with the community developmental health or health-related 
programmes; 

- and understand the simple administrative procedures, either routine or specific, within 
a primary health care setting; 

- how to identify, analyse and write reports on the problems of a community; 

- how to find solutions and prioritise them in solving problems of the community; 

- how to work with a health team in a community; 

- how to initiate, maintain and constantly evaluate established health programmes in a 
community. 

These objectives are intended to help the students to acquire the necessary knowledge, 
skills and attitudes for community health practice. 

Procedures Adopted in Preparation for the First Field Posting Experience 
A. Establishment of the Faculty Committee 

The Faculty Committee on CBME & SP was established in order to create a viable 
integrated programme which would meet the students' learning needs in the community. 
The Committee was charged with responsibility for: 

- developing suitable minimum learningobjectives; 
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- developing the strategy to meet the learning objectives, including community and 
sectoral involvement; 

- working out the requirements for carrying out the programme: 

- developing guidelines for the selection of suitable sites; 

- setting the date of commencement and supervision period of the field training 
programme; and 

- developing evaluation procedures for the prograrnrue. 

The membership of the Committee included representatives from all the depa* unents in the 
Faculty of Medicine; the Department of Sociology and Demography in the Faculty of 
Social and Management Sciences; participating communities; the students; Local 
Government; the State Ministry of Health; the Ministry of Education; and a general 
medical practitioner/community physician. 

The Committee was given the power to co-opt anyone who could contribute meaningfully 
to the success of the programme. 

B. Preparation of Staff 

The conduct of the field training programme rests on the staff of the Faculty, as well as the 
health institutions which serve as the Held or operational base for the students. Preparation 
of staff is here reported under three headings: seminar and workshops for Faculty staff, and 
preparation of field staff. 

Seminar for Faculty staff, A one day introductory seminar was organist for both academic 
and non-academic staff of the Medical School and the Department of Sociology and 
Demography. The objectives of the seminar included: 

- introducing the staff to the philosophy of CBME & SP and the nature of CBME & SP; 

- the method of implementation; 

- the roles of academic and technical staff in the implementation of the programme. 
Several papers dealing with different aspects of the community-oriented approach to 
medical education were presented. The presentors included a physician who was a 
graduate of the Community Medical Education Services (COBES) of the University of 
florin. 

Follow-up workshops. Subsequently, two workshops were conducted to discuss the 
involvement of the staff at every phase of the programme: 

1. Classroom involvement including lectures in community-related and social science 
subjects, and supervised discussion groups. 

2. Organisation of introductory orientation and educational visits. 

3. Execution of the programme in the field: 

- discuss the programme with staff to lead each team and live with the students 
throughout each field posting; 

- lead the students to their community location, including visits to social service 
institutions; 

- procure suitable accommodation for students and staff for each posting; 
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* guide students (if necessary) in planning their daily activities; 

- see to the welfare of the students through counselling when necessary; 

- supervise the programme; 

- liaise with the Faculty/Dean's office as occasion may demand. 
Preparation of field staff. Faculty staff and student representatives (2 males and 2 females) 
visited the health and educational institutions to be used as the operational base by the 
students: 

- to explain the Faculty's proposal on the CBME & SP field posting, and the benefits 
which could accrue from the interaction between the field institutions and the Faculty; 

- to discuss the roles of the local health staff. 

C. Preparation of the Medical Students 

The students must be the focus, if the community-based field training programme is to be 
successful. The preparation of the students must ensure: 

- a proper understanding of how to enter communities successfully; 

- familiarity with living arrangements in the community during the field posting; 

- understanding of the environment for working and learning in the communities; 

- appreciation of students working together as a team. 

To achieve these objectives, it was necessary to expose the students to a number of relevant 
preparatory and introductory topics on sociology, demography, health education, ecology 
and tropical environmental studies, human nutrition, epidemiology, statistics, microbiology 
and parasitology. These introductory topics were taught along side the orthodox basic 
medical sciences subjects (WHO, 1970). 

Orientation visits were made to places of interest in relation to community development 
and health. Principles of questionnaire design were taught, and the students were 
encouraged to put this into practice. The students were then given the minimum learning 
objectives as described by Alausa (1988). 

The students were also given intensive preparation in subjects, such as community 
dynamics, in order to develop appropriate attitudes and skills for successful entry into the 
communities with a good working relationship. In order to attain these objectives, the 
following steps were taken during the ten week preparation: 

1 . Lectures in basic medical science subjects, i.e. anatomy, physiology and biochemistry. 

2. Introductory/preparatory lectures in sociology (20 hrs), demography (18 hrs), health 
education (24 hrs), tropical environmental health (30 hrs), human nutrition (24 hrs), 
epidemiology and statistics (20 hrs), microbiology (8 hrs), and parasitology (10 hrs). 

3. Group discussion was also used extensively to explore the students' perception of 
society and the programme. 

4. Practical demonstrations featured prominently in the art and methods of communication, 
and methods how to conduct inverviews in a community, how to prepare a questionnaire 
as a major survey instrument, and how to complete the questionnaire. 

5. Orientation visits were paid to: 



73 



- some health delivery institutions within Kano metropolis, i.e. health centres, health 
clinics, dispensaries; 

- the Local Government Health Office; 

- the Water Supply Organisation known as WRECA; 

* refuse disposal sites in an urban and a rural community; 

- the market place. 

6. Student representatives accompanied the Faculty staff when preparatory visits were 
made to community leaders, i.e. the Local Government Chairman and H^kimi - the 
village and ward heads. 

£>. Joint Meeting of Staff and Students 

A joint meeting of students and staff took place approximately three *.veeks before the 
commencement of the field posting. The meeting provided an opportunity for questions 
and answers, particularly on the logistics. Potential problems were exhaustively discussed. 
The items required for the posting were identified by staff and students. They included: 

- academic/technical and administrative staff, 

- accommodation for resident staff and students, 

- beds, matresses and bedding, 

- gas lamps or lanterns/electric bulbs, 

- gas cylinders or kerosine stoves, 

- folding writing tables and chairs, 

- buckets and cans for drinking water, 

- cutlery and crockery, 

- suitable functioning vehicle with oil and petrol, 

- stationery, 

- calculators, 

- audio-visual equipment and public address system, 

- money for team leaders for incidental expenses. 

Request was made to the Faculty for the purchase of the above materials, except for a 
vehicle because of its prohibitive cost. 

E. Students Organisation and Preparation 

As the first field posting was to be based at Bichi and Dawakin-Tofa communities, the 
students were divided into two groups, each with 8 male and 3 female students. The 
students were encouraged to organise themselves into functional groups to manage their 
field activities, and the following groups emerged: 

- A map group which was charged with the responsibility of conducting a brief survey 
of the community, and to draw a map showing the more important places; 

- A programme development group was to design an appropriate questionnaire for the 
survey, and to plan the day-to-day activities. Staff assisted when necessary. 

- A food and welfare group was responsible for carrying out the domestic chores, as well 
as preparing the daily food. Rotation of groups and activities was agreed, in order to 
enhance the sharing of experiences. 

- A report writing group ensured that each team member, as well as the other students 
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would play a spcciSc part in the collation and analysis of the data which was fed into 

the group report at the end of the posting* 
Pre-course reading and pre-field posting questionnaire. The students were given some 
references from past COBES reports from Ilorin, as well as books and journal articles 
relating to community-based programmes. A questionnaire was completed by the students 
on the day before the programme commenced. The questionnaire asked for the following: 

- sex and ethnic background of the student; 

- what the student understood community dynamics to mean; 

- what the student understood CBME ASP to mean; 

- what the students expected from the community to which they were about to go; 

- what importance the students atteched to their preparation for the CBME & SP field posting; 

- whether the preparation had been adequate ; 

- what the students expected to gain from the field posting; 

- if the students anticipated any problems in the field, to list those problems, and to propose solutions; 

- what the students would like to do in addition to the Faculty guidelines; 

- what each student would like to see included in the preparation, in order to improve future postings. 
F. Preparation of the Communities 

Approval from Kano State Authorities. Any programme had to be approved by the State 
and Local Governments, in order for it to gain community acceptance. The appropriate 
authorities were approached by letter and personal contact; including the Military 
Governor's Office, the State Ministry of Health, the Ministry of Education, the Ministry 
of Social Welfare, and the Ministry of Local Government. Similar approval was sought 
from relevant parastistals or Government Agencies* 

Selection of the communities. The Dean of the Faculty gave (me of the authors (M.K.O.) 
the responsibility of designing the selection of practice sites for the students. A 76 point 
rating scale was designed (Fig. 1). 

Three Committee members visited 21 communities. Two communities with maximum 

scoces were selccicd for the first posting : Bichi, the headquarters of Bichi Local Government, 

andDawaki, thebeadquartersof Dawa 

with similar ratings were selected for the second fteld posting. 

Information gathering and dissemination. After obtaining the necessary approval from the 

relevant government agencies, a three-men committee visited: 

- 21 communities to see the community leaders to explain the establishment of Bayero 
Medical School and the CBME & SP philosophy, and to seek the cooperation and 
participation of the communities; 

- all the health institutions involved in Primary Health Care services to study their 
facilities. 

The opinion of community leaders and health staff were sought in relation to the health 
problems of their communities and the anticipated benefits that might accrue through this 
model of training of the undergraduates. 

Community participation. Follow-up visits were paid to the two communities that had been 
selected by the Faculty. Each community generously provided accommodation at their 
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own expenses for staff and students* The cost of water and electricity was also met bv the 
people* 

G. Final Phase of Preparation 

The minimum learning objectives were given to the students, and one staff member was 
asigned as leader to each team, while the students were in the field. Staff with specific skills 
were appointed to visit each team twice weekly in the first two weeks of the programme 
and, thereafter, once a week until the end of the posting. The visits were intended to help 
staff and students to solve problems which the resident staff might not be able to resolve. 
These visits would also have a morale boosting effect 

Start of the Field Programme. The familiarisation visit took each group to the head of their 
local government district and the counsellor for health (Fig. 2). The District head ensured 
that places of interest to the programme could be visited without hinderance. More 
importantly, a ward head was assigned to the students to take them round and introduce 
them to the communities. Visits were also paid to places such as the primary and secondary 
schools, the native court, the market, the bank and police station, and recreational facilities 
in each of the villages. With the map and the questionnaire that the students had prepared, 
they began to probe into the demographic characteristics of the communities and their 
health problems. 

Discussion 

The Bayero Medical School, a newly established Faculty, had no tradition to follow. Its 
newness was an advantage for commencing a programme without traditional opposition 
from staff; hence the successful entrenchment of the CBME & SP into the curriculum. The 
idea of a community-based programme was new to almost all the staff. The exceptions 
were the dean, one of the guest lecturers who was a community physician, and the author 
(M.K.O.), who was given the challenge of organising the programme. The very newness 
of the concept became a motivating factor for both staff and students. The students 
accepted the programme without reservation. Kano State was at that time badly in need of 
trained health professionals, particularly physicians. The idea of cooperation and 
collaboration between selected communities and the medical school was most welcome, 
as it gave the people hope of solving their need of health professionals. 
The thoroughness in preparing the programme in just 10 weeks, the use of defining 
objectives and the methods adopted for preparing the students about the dynamics of 
communities, and the way in which the ideology was presented to the staff during seminar 
and workshops were essential elements in the successful introduction of the programme. 
Furthermore, careful observation of the offical government protocols and the numerous 
personal visits to agencies and community leaders contributed immensely to the smooth 
acceptance of the students into the communities. The cooperation of the University 
authorities was commendable. 

The major constraints during the period of preparation were fourfold: 
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- Acute shortage of academic staff. The effect of staff shortage on supervision of students 
is well known. 

- Lack of staff with previous knowledge and experience in community-based program- 
mes made considerable demands on the main organiser of the programme. 

- Lack of vehicles was a major constraint 

- Gross inadequacy of books and journals had to be faced. 



I. Points Well defined cofmnunity(ies) with similar socio-cultural background: 

1 (a) Not well defined community (ies) 

(6) 2 (b) Well defined but non-homogenous - language/religion 

3 (c) Well defined and homogenous 

n. Availability of health facilities - community/government: 

0 (a) None available 

1 (b) Availability of Health Faculty 
(10) 2 (c) Available and functioning 

3 (d) Private or community-owned 

4 (e) Operating with qualified staff 

III. Rural Health centre infrastructure: 

0 (a) None 

1 (b) Available 
(6) 2 (c) Functioning 

3 (d) Located less than 1 Kilometer from the community 

IV. Accessibility - good road / nearness to the Medical School: 

1(a) Footpath 

(6) 2 (b) Unsealed but negotiable 
3 (c) Sealed road 

V. Receptiveness of the community(ies): 

1 (a) Poorhospitability 
(6) 2 (b) Indifferent 

3 (c) Hospitable/warm 

VI. Population / manageable size of household: 

1 (a) Fewer than 2,500 (200 houses), not above 15,000 (1500 houses) 
(6) 2 (b) Between 2,600-5,000 (260-500 houses) 
3 (c) 5,000*14,999 (1400-1500 houses) 
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VII. Availability of suitable accommodation for students/staff: 

0 (a) None available 

1 (b) Available for 15-20 students 

(10) 2 (c) Accommodation capable of taking tudenU and staff. 

3 (d) Available with adequate windows and toilet facilities 

4 (e) Available with extra space, e.g. for group meeting and recreation 



VIII. Cost of accommodation: 

1 (a) Available for rent to Faculty 

2 (b) Available at partial rent to Faculty 
(10) 3 (c) Available free to Faculty 

4 (d) Payment of electric bills by the community 



IX. Well / piped water supplies: 

0 (a) None available 

1 (b) Available sporadically 

( 10) 2 (c) Continuously available but not in sufficient quantity 

3(d) Available always in adequate quantity and quality 

" 4 (e) Available with good alternative supply in clean environment 



Availability of electricity: 



0 (a) None available 

(6) 1 (b) REB (Rural Electrification Board) 

2 (c) National Grid 

3 (d) National Grid and standby generator 



Maximum score points: 76 



Figure 1. Scoring scheme for selecting communities, devised by M.K. Omonisi 



Conclusion 

The challenge of preparing staff, students and the communities for the programme was met 
successfully because of the full cooperations by all concerned, and the commencement of 
the programme was thus very encouraging. The solution of the problems that were 
identified would need a number of decisive interventions. These would include: 

- Recruitment of interested and committed physicians and other health professionals 
with adequate experience in running community-oriented programmes; 

- Employment of Kano indigenes to participate in the programme; 

- Incentives for staff who participate in the programme; 
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- Purchase of new vehicles or repair of existing stock; one bus would be ideal for each 
site, each equipped with a public address system; 

- Purchaseofappcopriate textbooks and^ 
and implementation of the programme. 
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Activities 



Monthi 
Wceki 
Days 



March 



1 2-6 



g 

8-14 



m 



15-18 19-2122/3 24-27 



IV 



28-34 



April 
VI 

135*3 37-41 



Introduction to 
community and 
leaden 



Operational 

activities/ 

planning 



Pretesting of 
instruments 



Data collection; 
Primary project 



Data analysis 



Instrument design 



Optional project, 
data collection 
and analysis 



Report writing 



Presentation of 
report to Faculty 
and written exam 



Figure 2. Schedule of the field training programme 



Annals of Community-Oriented Education 
1992, Vol. 5, 81-95 



81 



Training Students in the Principles of Community-Oriented 
Primary Care in the Rural Southern United States 

Suzanne E. Landis, Harry Summerlin and P. Richard Olson 

University of North Carolina 

Chapel Hill, North Carolina, United States of America 

Abstract 

We report on the development of a four-week rotation for fourth year undergraduate 
medical students and third year postgraduate family practice residents based on the 
methodology we have used in an ongoing Community Oriented Primary Care (COPC) 
Project located in Madison County, North Carolina. The COPC project has: defined and 
characterised the county; identified county health issues; modified or developed new 
healthcare programmes to address priority health issues; and is monitoring the effectiveness 
of these programmes. The student experience is planned to run in parallel with the above 
process through the use of structured self-directed learning tasks, interviews with 
community people, anddenominator-data-informedclinicalpatient experiences ahng 
physicians and non-physicians. Students learn what data to gather about a community, 
where to access this information, how to define perceived health problems with people in 
the community, how to work with community groups, and how to treat patients within this 
community perspective. Only through such learning experiences will students develop the 
attitudes and skills they need to practise medicine with a community perspective. 

Introduction 

To produce primary care physicians who are better able to promote healthy lifestyles and 
improve the community's health, we must adapt undergraduate medical education and 
postgraduate medical education (residency training programme) to systematically provide 
learners with instruction in population-based skills such as the process of Community- 
Oriented Primary Care (COPC) (Abrahamson, 1983; Institute of Medicine, 1984; Nutting 
et al., 1985; Nutting, 1987). COPC can be used as a potential model to outline the 
experiences students (undergraduate medical students and postgraduate residents) need to 
participate in caring for a community, as well as for individuals (Pickles, 1939; Lipkin et 
al., 1982; Hess et al., 1983; Prywes, 1983; Strelnick et al., 1986). 
The qualifications necessary to participate in the COPC approach include: an interest in 
improving the community's health; the ability to define "community** through collection 
of vital statistics, the identification of the community's perceived health needs, visits to 
churches and community meetings, and visits to politicians, law makers and law enforcers; 

Requests for reprints should be sent to Suzanne £. Landis, Mountain Area Health Education Center, 501 
Biltmore Avenue, Asheville, North Carolina, 28801 4686, United States of America. 
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a willingness to collaborate with community members in health-related issues; and an 
understanding of epidemiological principles (Fraser, 1987; Mullan, 1984). 

Student COPC Experiences 
County Demographics 

A rotation for fourth year undergraduate medical students at the University of North 
Carolina at Chapel Hill and third year Family Practice postgraduate residents at a 
University of North Carolina Family Practice Residency Program in Asheville, North 
Carolina is located at the Hot Springs Health Program (HSHP) in western North Carolina, 
a five hour drive from Chapel Hill and a fourty-five minute drive from Asheville, North 
Carolina. The HSHP is a comprehensive, county-wide health programme that provides 
primary medical and dental services to all the people of Madison County. The HSHP 
operates four primary medical centres, a dental centre, pharmacy services, a home health 
agency, and hospice services. The six physicians, one family nurse practitioner, one 
physician's assistant, IS home health nurses, one dentist and one pharmacist provide 
services at the medical centres, the health department, the two nursing homes in the county 
and the local college infirmary. The physicians, the nurse practitioner, and the physician's 
assistant provide care on a rotating basis at all four medical centres, the health department, 
the nursing homes and the college infirmary. The medical centres are open during the 
evening and weekend for routine care, and the medical providers are available by phone 
at other times. 

Madison County is a mountainous rural county with approximately 17,000 people, 
essentially all Caucasian. No large industries exist within the county borders. People are 
involved with mall plot farming, especially of tobacco, or employed by a local liberal arts 
college, the HSHP, or the local utility company. Many people also work outside the county. 
Fifteen percent of the population is over the age of 65, compared to the national rate of 
11.2%. Forty percent of the elderly live below the poverty level. Five percent of the 
population is unemployed. 

Development of the Student COPC Experience 

In July, 1989 representative of the HSHP, the local liberal arts college, and the Family 
Practice Residency Program in nearby Asheville, received funding from the W.K. Kellogg 
Foundation to develop a Community-Oriented Primary Care Project in Madison County. 
A major focus of the award was the development of community-based arid community- 
operated primary prevention programmes and the implementation of a student COPC 
experience. 

Objectives of the Student COPC Experience 

The main objectives of the student COPC experiences are to: 1) recognise the manife- 
stations of disease in a community, its effects on individuals, their families and the 
community, and the available resources/services for these diseases; 2) develop a sense of 
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"community" through first-hand experience with the people, community groups, services 
agencies, and through the acquisition of community-wide epidemiological data; 3) learn 
how to interact with community agencies or boards, in order to sort health needs, develop 
and monitor programmes; 4) apply the definition of "community" and the knowledge of 
available services to the individual patients seen during the rotation; and 5) complete a 
Community Project 

Implementation of the Student COPC Experience 

Table I outlines the experience as it relates to the COPC process. The student defines and 
characterises Madison County by: answering specific demographic questions and morbidity/ 
mortality questions; interviewing community people about their perceived health problems 
and needs; visiting health agencies, churches, and firefighter and community development 
club meetings; and talking with recognised community leaders and natural helpers. 

Next, the student summarizes and prioritises the health problems and selects one to 
evaluate intensively. Then, to demonstrate the value of this population/community-wide 
information (the denominator), the student performs acase study with a patient representing 
this health problem from the HSHP practice. Information on the patient and family (the 
numerator) can be seen in relation to the community's experience with the medical 
problem. 

The student then designs a hypothetical intervention or works with the Community 
Advisory Board on an ongoing intervention. If time permits the student can be involved 
with collecting evaluation data of an ongoing intervention. These interventions may be 
located within the HSHP or in the community, depending on the nature of the medical 
problem. 

Half of each day is spent working with a physician in one of the four medical centres, the 
health department or the nursing homes. The other half day is spent collecting information 
about the community through existing sources of data, community opinion polls, visits 
with home health nurses to individual patients 9 homes, visits to service agencies and 
community meetings. At the end of the rotation, the student is expected to submit a written 
report that includes a description of the community and the existing health care system, a 
problem as perceived by the community or student, and a proposed plan of intervention for 
solving this problem, i.e. the Community Project. 

A workbook has been developed to aid the students in their learning. The workbook 
includes: basic logistical information as to contact people, housing and directions to the 
site; objectives of the rotation; a tentative weekly schedule; a specific course outline; and 
student evaluation forms. 

Defining and Characterising the Community 

The course materials provide a list of questions that define and characterise the community 
using demographic data, morbidity and mortality data, patient opinion polls, available 
health care services, existing community groups, natural and elected leaders. Specific data 
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sources, persons to be interviewed, and specific questions to answer are provided for the 
student. 

Clinical experiences follow a standard ambulatory care format; first the student interviews 
and examines patients, followed by one of the HSHP physicians examining the patient for 
confirmation. The student also gathers information about the community and is expected 
to complete a Community and HSHP Practice Worksheet* 9 (Fig . 1 ) summarizing the age 
distribution, gender, risk factor profile, and leading medical problems within Madison 
County, within the Hot Springs Health Program practice, and among the patients 
personally seen by the student. This worksheet provides at a glance the similarities and 
differences among the patients seen by the student and those within the HSHP as a whole. 
County information is available through: a state-provided statistical summary entitled the 
County Data Book; a limited opinion poll conducted by the student with five patients at 
the HSHP and five patients at the County Health Department (Fig. 2 for the questions); and 
the Centers for Disease Control Behavioral Risk Factor Survey results for Madison 
County, listed in the County Data Book. The HSHP patient data is available through the 
computerised patient database; students have the opportunity to learn how to access age, 
gender, risk factor, screening preventive services, and diagnoses in the HSHP practice as 
a whole. Individual patients seen by the student will provide the comparative information 
required to complete this analysis. 

Identifying the Community Health Issue(s) 

After summarizing this information, the student will attend one or more meetings of the 
Community Advisory Board to watch the interaction between agency personnel and 
community lay people and to participate in the discussion of ongoing and newly developing 
interventions. The student then selects one health issue identified through his synthesis of 
the epidemiologic data and the opinion poll, an issue that is also of personal interest. The 
student describes the magnitude or scope of this health issue by estimating: the prevalence 
and/or incidence of the problem; the nature of the impact of this problem on rates of 
morbidity, mortality, disability, and hospitalisation rates; direct, indirect or intangible 
costs to individual patients with this problem; and the community's societal costs. Finally, 
the student comments on the existing types of health services available in the community 
to address this problem and current obstacles to using this health resource(s). 
The student then selects a patient with this health problem from the HSHP practice through 
the use of the computerised problem-centred database. The student performs an in-depth 
case study through visits with the patient/family at home and at the workplace. The student 
also visits the agencies which the patient uses, to describe how the patient and his care 
exemplifies this iiealth problem. The student will discover the chief complaint, the past 
medical history, family genogram, and the patient's health status (functional status, ri*!: 
factors for the disease, health elated behaviours, compliance with health care directives, 
and use of health services in the commir ./). The case study exemplifies for the student 
how knowledge of the community, both from an epidemiological ami from a sociological 
perspective, can add to the care for an individual patient 
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Madison 
County 


Hot Springs 
Health Programme 


Students/Residents 
Patients 


Age 

Mean (yrs). 
< 1 yr. 
1-18 
19-65 
>65 








Gender 

M(%) 
F(%) 








Risk factor 

% Smokers 

% High Blood Pressure 
% Hypercholesterolemia 
% Seat Belt Users 
% Sedentary Lifestyle 








Leading Medical 

Diseases/Problems 

1. 

2. 

3. 

4. 

5. 









Figure 1. Characteristics of community and practice worksheet 



Modifying or Developing New Health Care Programmes (or Interventions) 
Then, based on the analyses of the community, patient and family, the student proposes 
specific solutions, interventions or treatment strategies for dealing with that health 
problem. These solutions must be appropriate for the community and must rely on 
feedback from the physician-mentor, health agency personnel, elected or appointed local 
officials, and community lay people. At any one time the Community Advisory Board is 
involved in two or three intervention programmes, and the student may elect to participate 
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1 . What personal health problems or physical complaints appear to be most commonly 
mentioned by people in the community? 

2. What barriers to health care or medically-related issues do people in the community 
talk about? 

3. What in your opinion are the serious health problems in the county; 
What are the causes of these problems; and 

What in your opinion are the serious barriers to meeting these health needs? 

4. What needs to be done to handle these problems? 

5. Do you feel that there is any group of the population not receiving adequate medical 
care? Why not? 



(This questionnaire may be reproduced as many times as needed for interviews). 



Figure 2. Patient interview questions about health needs and services in Madison county 

in the planning and implementation of some aspect of these interventions. The student 
synthesizes these findings and proposals into a typed eight to ten page report, known as the 
Community Project. A sample report is included in the workbook. 

Logistics of the COPC Rotation 

This COPC curriculum was designed during 1989 and 1990, and the first students were 
welcomed in the Summer of 1990. The University of North Carolina School of Medicine 
provides undergraduate medical training for 160 students in each year of the four year 
course. The first two years provide basic science training and the third and fourth years 
provide clinical experiences. The COPC rotation is an optional four week rotation for 
fourth year students . The HSHP felt that with the current workload for their phy sicians they 
could provide adequate teaching for only one student a month for a total of twelve students 
a year. 

We estimate that one student requires twenty percent of the physician* s time. In addition, 
the community opinion poll consumes up to thirty minutes for each of ten patients; the 
visits to health agencies, such as the Department of Social Services or the Mental Health 
Center, and visits with community leaders require at least one hour per visit We recognise 
thatcertainhealth agencies and community leaders could potentially become overwhelmed 
with student discussions; to avoid this we suggest certain agencies and leaders on a rotating 
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basis, so that no one person is called upon more than once a year. Hie community opinion 
poll is conducted on ten randomly selected patients per month; it is unlikely that the same 
patients will be asked more than once in any one year. 

In July of 1991 we started to offer this COPC experience to postgraduate Family Practice 
residents in their third year of a three year Family Practice Residency Programme. We have 
thus one undergraduate fourth year medical student and one third year resident present 
simultaneously. Residents in the ; - final year require minimal supervision from the 
physician-mentor in comparison with undergraduate medical students. 

This COPC experience is administered by the Hot Springs Health Program. Students live 
either at a local inn within the county or in the dormitory of the local liberal arts college 
during their four-week experience. They are required to pay for their travel expenses within 
the county and for their meals. 

One of the physicians is responsible for introducing the student to the practice and 
assigning one physician-mentor to the student to work with on a daily basis. The physician- 
mentor, with the assistance of the Data Management Supervisor for the HSHP is 
responsible for training the student in the use of the computerised database. Currently, the 
W.K. Kellogg Foundation is providing financial assistance to the HSHP for the 
implementation of this COPC experience. The University of North Carolina provides the 
reimbursement for housing and travel to the site. 

The student completes the self-directed learning tasks in the workbook each week and 
reviews these with the physician-mentor. The student's final grade of Honors, High Pass* 
Pass or Low Pass is determined by the mentor commenting on the clinical patient care 
activities (50%) (Fig. 3), the written Community Project (40%), and the final oral 
presentation of the Community Project to the HSHP (10%). The student is expected to 
complete an evaluation of the rotation by commenting on the overall benefit of the 
experience, the quality of the preceptor's help, and the value of the Community Project 
(Fig. 4). 

Community Responsiveness 

Woven through this student experience is the persistent theme of striving to know the 
community through the collection of existing health care statistics, perceived health needs, 
individual and family case studies, and the personal interaction with health agencies/ 
personnel, the Community Advisory Board, and people in the community. The experience 
of gathering information and interacting with community agencies and people offers the 
student a broad perspective in which to place the health problems and needs of their 
individual patients during this selective, i.e. the denominator view rather than merely the 
usual numerator view of medical problems. This experience of clinical patients informed 
by denominator-data is a unique opportunity for the student to understand tht value of 
gathering such information in order to help an individual patient. 
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These experiences have the potential to benefit the patient, family and the community. The 
patient and family can benefit through a more expedient use of ancillary health services and 
through more appropriate medical referral. The community can benefit when a health care 
provider promptly recognises epidemics, notices gaps, or redundancy in medical services, 
and thus provides a more streamlined medical care delivery system. 

The students can provide valuable information to the Community Advisory Board. This 
Board is already informed about the county epidemiological health data and the results of 
a large community and health professional opinion poll conducted in the fall of 1989. 
However, the students can update the Board on the changing perceptions of health needs 
from their limited patient opinion poll, the results of the patient case studies, and their 
discussions with community leaders and groups. The students' recommended solutions/ 
interventions for the selected health problem are reviewed by the Board and used when, 
and if the Board implements a programme to address that health problem. The students are 
actually a part of our overall and ongoing evaluation of the COPC process within the 
county. 

The students begin to understand the views of community leaders or agencies who will 
support or hinder changes in, or the development of new health programmes. Through 
discussion with various community people, community groups, and the Community 
Advisory Board students come to recognise the potential difficulty and perhaps the value 
of becoming involved in community policy-making. 

Conclusion 

We have developed in a rural Southern United States mountainous county a COPC Project 
incorporating the primary health service delivery system, the Hot Springs Health Program, 
and a newly composed Community Advisory Board. The Community Advisory Board 
sorts health problems and designs/implements community-based and HSHP-based 
interventions. Students are able to experience the COPC process through specific self- 
directed learning tasks and interactive sessions with health care professionals in the HSHP, 
members of the Community Advisory Board, and lay people in the community. The 
students' experience, including both patient clinical experiences and the Community 
Project, has been designed iu Hi into a standard four -week time period. 

This COPC experience consists of direct patient clinical experiences within the framework 
of learning about the health of the whole county. Additionally, the experience provides 
practice with the principles of COPC. Students learn what data to gather about a 
community, where to access this information, how to define perceived health problems 
with people in the community, how to work with community groups, and how to treat 
patients within this community perspective. Only through such learning experiences will 
students develop the attitudes and skills they need to practise medicine with a community 
perspective. 
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Student name: 
Date of elective: 



Mentor name: 



Considering the student's stage of training, rate his/her development with respect to each of the 
following areas. Please make an; > detailed comments below each item. 

1. Ability to collectdata pertinent to diagnosis and management ofcommon problems in ambulatory 
patients, their families and their community. 



Consistently obtains 
necessary information 
through patient inter- 
view, examination and 
appropriate use of tests 



4 3 

Frequently Occasionally 

obtains all obtains all 

information information 



2 

Seldom 
obtains 
all infor- 
mation 



1 

Consistently 
unable to 
obtain perti- 
nent data 



Comments: 

2. Knowledge and skills in the primary management of common problems in ambulatory patients. 



Consistently able 
to evaluate and 
plan management 
for patient problems 

Comments: 



Usually 
able 



Occasionally 
abl 



Seldom 
able 



1 

Consistently 

demonstrates 

inadequate 

evaluation and 

management 

ability 



3. Evaluation, education and treatment of patients and their families, utilising knowledge of socio- 
economic and psychological factors. 



Always manages patient Ususally Often done Seldom 
problems comprehen- done done 
sively 

Comments: 

4. Ability to provide continuity of care for patient problems, when needed. 



Established and carries 
out appropriate follow- 
up for all or almost 
all problems 



4 

Usually 
done 



Often done 



2 

Seldom 
done 



1 

Displays a 
consistent dis- 
regard for 
aspects of 
patient care 



1 

Consistently 
unable to pro- 
vide follow-up care 
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Comments: 



S. Quality of function within the health care team in the community. 



4 

Exhibits 

good 

ability 



3 

Exhibits 

satisfactory 

ability 



2 

Exhibits 
marginal 
ability 



1 

Functions 
poorly with 
the other 
members 



Exhibits superior 
ability to participate 
in patient care with 
nurses, staff, and 
community agency 
representatives 

Comments: 



6. Utilisation of medical resources in your area, including consultants, hopitals, social agencies, 
paramedical personnel, etc. 



Shows superior 
ability to select 
and utilise other 
resources, when 
needed 



Shows good Shows satis- Shows mar- Poor recog- 
ability factory ginal ability nitionofneed 

ability and utilisation 



Comments: 

Following your four weeks of precepting, you and others in your practice have a unique opportunity 
to evaluate the student's clinical and personal abilities in your setting. 

7. What are the student's particular strenghts? 

8. What areas could use some improvement? 

9. If you would like to recommend this student for the grade of "honours", please attach * letter 
describing your specific, detailed reasons for judging him or her outstanding. Also, attach a letter 
detailing your reasons for an "unsatisfactory" grade. 



Signature of mentor completing 
the above evaluation form 



Figure 3. Community-oriented primary care evaluation by mentor 
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Date of Evaluation 



Student Name 



Your mentor and the COPC faculty are interested in your perceptions of the clerkship you have just 
completed To aid us in maintaining or improving the quality of your educational experience, your 
candid assessment of your experience is a valuable tool in evaluating the programme. 



1. Overall, how much medical 
knowledge did you gain 

2. Was it beneficial to learn 
about the community in mch 
one is to practise medicine 



Very Much Much Moderate Little None 



1 



How often did you: Always 

3. Get involved in patient 1 
management 

4. Receive supervision from 1 
your preceptor 

5. Receive teaching or in- 1 
struction from your zpreceptor 



Usually Sometimes Seldom None 
2 3 4 5 



Definitely Probably Not 
Yes Yes Sure 



Probably Defini- 
No tely no 



6. Knowing what you do 
now, would you recommend 
this rotation to a classmate? 



How would you rate your perceptor's help to you with respect to the following areas? 

Excellent Good Average Fair Poor 

7. Understanding the process 1 2 3 4 5 

of preventive medicine and 
health maintenance 



8. Understanding the roles of 
family members in health, 
illness and patient management 
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9. Understanding the need for 
continuity of care in patient 
management 

10. Understanding the need 
for continuing education after 
you are in practice 

1 1. Understanding of the 
patient referral process 

12. Understanding the econ- 
omics of office practice 

13. Understanding the role of 
community agencies (e.g. 
Health Department and Depart- 
ment of Social Services) in 
the practice of medicine and 
provision of health services 

14. Increasing your ability to 1 2 3 4 5 
identify and use community 

agencies in patient manage- 
ment 



How would you rate your rotation with respect to the following areas? 

Excellent Good Average Fair Poor 

15. Amount of time your pre- 1 2 3 4 5 
ceptor made available to 

you to discuss aspects of 
the clerkship 

16. Amount of feedback you 1 2 3 4 5 
received from your preceptor 

on the quality of your per- 
formance and/or possible 
ways of improving your skills 
and knowledge 

17. Variety and quantity of pri- 1 2 3 4 5 
maiy care conditions obser- 
ved during the preceptorship 
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1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
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18. Amount of time spent 1 2 3 4 
working with health 

professionals in the 
community health agencies 

19. Opportunity to discuss 1 2 3 4 
current local health care 

issues with your preceptor 

If you wish to qualify any of the above answers, please use the following space. 



20. If you were ready to enter medical practice, list in order of importance those factors which you 
and your family would consider regarding a practice in this community. 



21. What do you consider to be the strong points of this educational experience? 

A. 
B. 
C. 
D. 



22. What do you consider to the be weaker points of this educational experience? . 

A. 
B. 
C. 
D. 



23. How did this experience compare educationally with your previous medical training? 



24. What suggestions would you make for changing this clerkship? 



Figure 4. COPC selective evaluation 
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Organisation of a Community-Oriented Faculty 

Richard G. Christiansen, Joseph F. Boero and Mohan L. Garg 
University of Illinois at Rockford 
Rockford, Illinois, United States of America 

Abstract 

A community-oriented medical school is described where the entire traditional hospital 
clerkship curriculum is provided by trained, committed, community physicians and ail the 
out-patient community practice curriculum is provided by fidl-time academic physicians 
operating in university-owned practices. The focus of the paper is on the part-time 
teachers. We suggest that there are physicians in every community, who, if recruited, can 
be trained to create and teach a medical school curriculum. The attitudes of these 
physicians toward salary for teaching and toward the importance of teaching in their 
careers art discussed. 

Introduction 

The past sixteen years have afforded us the opportunity to observe a medical education 
programme that uses full-time academics for out-patient experiences and part-time 
community physicians (salaried and non-salaried) for inpatient clerkships. We feel that 
this model has merit and may be applicable to other community-oriented medical schools. 

In Rockford, the outpatient experience is continuous and lasts 30 months starting in the 
second year and listing until graduation. The supervised students follow their own panel 
of patients. This means that families are assigned to the students, and, except for 
emergencies, the patients in that family are scheduled to see that student when they are sick. 
The hospital clerkship experience is 36 months in duration and begins in the second year 
with the Introduction to Clinical Medicine and Physical Diagnosis courses. The students 
are allowed to absent themselves from these courses and from the subsequent hospital 
clerkships for two half-days per week to participate in their outpatient clinics. 

Ourexperiencehasled to the following theses: 1) each community has a cadre of practising 
physicians who have the ability and arc willing to make the commitment to create and teach 
a medical school curriculum; 2) hospital curriculum can be taught totally by practising 
community physicians (who have access to practice in the hospital); and 3) continuous 
outpatient clerkships function well when taught by full-time academics. 
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The University of Illinois regionalised its medical education sixteen years ago. The 
original school remained in Chicago, and three additional sites were developed. The school 
in Chicago, with its yearly intake of 160 students, has maintained a traditional four year 
curriculum. The 140 students at the three other sites have their first year at Urbana and then 
either remain in Urbana or go to Peoria or Rockford. There are 50 students per year, for 
three years, at Rockford. 

The Rockford site is ISO km from Chicago and has a metropolitan population of 250,000. 
It is 350 km from Urbana and 250 km from Peoria. Rockford has three private hospitals, 
and all three have signed agreements to provide resources and patients for teaching. These 
hospitals are not owned by the University. The total teaching staff of the school consists 
of 400 physicians, but only 30 are full-time with the University. The Department of 
Medicine has eight full-time physicians. One is the Chairman of the Department and he 
remains in clinical practice within the community. Two are retired physicians from the 
community who have the time to give individual tutoring to (he students. The remaining 
five are assigned as practising physician-teachers in the university-owned out-patient 
clinics where the students have their 30 month continuous experience. The rest of the 
Department teaching staff, 130 in number, are part-time salaried or non-salaried members 
of the Faculty. 

Definition of Community Orientation 

The Rockford version of medical education uses community resources as its driving force. 
The community physicians develop and teach the curriculum. The community hospitals 
provide their facilities for teaching clerkships. Community leaders in government, 
religion, industry, business, social services, and education determine and teach that portion 
of the curriculum that is concerned with the skills needed to practise primary care medicine 
within a community* The University provides outpatient clinics, staffed by full-time 
academics, who give students an extended outpatient learning experience. 

Teaching Staff (Department of Medicine) 

The full-time faculty members, who teach in the university-owned community practices, 
where the medical students have their continuous out-patient clerkship, are career 
educators. Their focus is on the delivery of medical education in outpatient setting. 
Their large teaching and clinical service responsibilities make it necessary that they have 
a support team that assists them in advancing their academic career. This means that 
medical educators or clinical research specialists must be assigned to work with them in 
their clinical setting, much like bench laboratories would have research assistants, in order 
for significant academic productivity to go on. Their salary is a combination of university 
salary and monies from clinical practice. 
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The members of the part-time teaching staff fall into four categories depending on their 
level of commitment (Figure 1). The first group is classified as "educators" and are 
committed to learning and thinking about curricula 

re-mediation, etc. They function as departmental leaders and planners* The members of the 
second group are "committed teachers" - they allow students to be with them at all times 
during their practice and they give lectures, seminars, and patient presentation conferences 
whenever they are asked to do so. Those in the third group are "occasional teachers" 
providing at least one teaching function during the year. The fourth group is called 
"inactive". The Department's rules state that teachers can remain "inactive" for three 
years. After that time, they must either resign from the Faculty or again participate in 
teaching activities. 



Figure 1. The number of part-time faculty in each category 
Community Physician Commitment 

Experience has shown that it is necessary to appreciate a number of concepts before 
* developing a Faculty of committed community physicians. These are: 
1. Recognise from the beginning that there are varying levels of desire to teach among 
community physicians. Those that want to make a commitment need to be identified and 
recruited. The goals of the teaching programme and the opportunities that are available in 
the school need to be explained to them on an individual basis. Figure 2 demonstrates the 
results of a survey of salaried members of the Faculty. The 77% response shows that there 
are many who consider the medical school appointment important to their careers. This 
demonstrates that there is a cadre of practising physicians who feel that involvement with 
the medical school is important. 
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Figure 2. My teaching appointment is important for my career plans 



2. Job descriptions are necessary at every level of responsibility within the Department of 
Medicine. Clinicians need to know exactly what is expected of them, so that they can 
adequately judge whether their skills and practice styles fit these expectations. The 
following outline shows the positions that are available in the Department of Medicine and 
some of the job descriptions: 

Educator 

These faculty members will have one of the positions listed below. Each of the positions 
has a detailed job description. 

Discipline Division Chairman (Cardiology, Nephrology, etc.) 

Continuing Medical Education Coordinator 

Evaluation/Re-mediation Coordinator 

Research Coordinator 

Post Graduate Education Coordinator 

Physical Diagnosis Course Coordine'or 

Introduction to Clinical Medicine Course Coordinator 

Medicine Clerkship Coordinators. 

Committed Teacher 

a. May have students assigned for the entire year. 

b. Must participate, on a rotating basis, in the physical diagnosis curriculum. 

c. Must participate (when asked) in lectures, conferences or seminars. 

d. Must participate in at least one standing medical school committee. 

e. Must be responsible for regularly submitting and revising examination questions. 
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Occasional Teacher 

a. Must participate in at least one medical school or teaching assignment per year. 

b. The teaching assignment may be determined at the convenience of the Faculty member. 

Inactive 

a. A person may choose to cease medical school activity entirely. 

b. He or she may remain in this category for three years. 

c. After three years, if no activity is re-instituted, the faculty appointment will be 
terminated. 

3. Salary is important to private physicians who commit themselves to being educators or 
committed teachers (Fig. 3). The survey shows that a significant number of the salaried 
physicians consider payment for teaching activities to be a necessity. In addition, the 
signing of a yearly contract is essential. This ensures that the salary goes with the position 
and not with the person, and that the amount is renegotiated each year. This prevents all 
the income from going to persons with seniority, as the effect of the annual university 
salary raise, on each individual's salary, is eliminated. It is also important to have an 
"inactive** category for part-time Faculty members, because there are many other potential, 
yet temporary commitments in medical practice. For example, someone might be elected 
Hospital Staff President. The individual teacher has to have a dignified way to be released 
from teaching for a period of time. 



NECESSARY 5 ■ 130 



Figure 3. Is salary necessary? 

4. The educators among the part-time staff of the Faculty must design the curriculum and 
bring it to the committed and occasional teachers for their approval. Faculty teacher 
training sessions should be very practical, as the teachers want information and techniques 
that will help them with their day to day teaching responsibilities with students, not 
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educational theory. Working sessions function best at 7:00 a.m. for breakfast, or 6:00 to 
8:00 p.m. for supper. Longer training sessions or workshops begin at 10:00 a.m., include 
a working lunch, and finish by 2:00 p.m. This permits the physician to make morning and 
afternoon patient rounds. 

Cost of the Part-Time Faculty 

The Department of Medicine funding for teaching staff is about one-half of the total 
Rockford site Faculty budget The funding for the 1 30 part-timers shown in Figure 2 is one- 
third of the departmental budget and therefore one-sixth of the total Faculty budget We 
feel that a full-time staff, large enough to deliver our curriculum, would be far more costly 
for the School. There is no part-time Faculty member, even the most committed in the 
"educator" group, who earns more than 15% of his or her income from teaching; most earn 
less than 5%. We have found, that if someone accepts money, no matter how small the 
amount, for an important responsibility, such as educating medical students, the person is 
likely to take the task very seriou; ly. "Education" time, particularly for those who head 
divisions, or curriculum segments, can approach 10 to 20 hours per week. 

Student Outcomes 

After sixteen years of experience with our community-oriented teaching staff there are 
several questions that should be answered. Do the students perform as well in national 
examinations as those in a traditional curriculum, and do more of them go into primary care 
residencies? Figure 4 shows that our students perform consistently as well as students who 
attend the more traditional curriculum at the Chicago site of the College of Medicine. 
Figure 5 shows the percentage of Rockford students who have gone into primary care 
residencies each year since the first class graduated in 1 975 . The percentages arc compared 
with the USA national averages for the past seven years. The results show that there is no 
obvious influence of our curriculum on the choice of career by students. 

Conclusion 

In this model of organisation for acommuidty-oriented medical school part-time community 
physicians are recruited and trained to provide the leadership and teaching for a complete 
clinical curriculum. Most of the full-time physicians at the school arc concerned with 
teaching in a continous outpatient curriculum. The two groups of teachers work together, 
allowing students to pass back and forth between experiences, so that a coordinated, 
complete, community-oriented curriculum is formulated. This model allows the University 
to introduce relevant clinical material relating to community medicine without the expense 
of supporting large numbers of full-time physicians. The financial rewards for the 
committed part-time physicians arc small but the opportunity to participate meaningfully 
in the scholarly process and the prestige and satisfaction this brings is enough reward to 
ersure continued commitment. 




Figure 4. Comparison of fourth year clinical examinations between Rockford and a 
traditional school (U of I at Chicago) 
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figure 5. Percent of Rockford students entering primary care residencies compared to all 
Ajnerican graduates (1985-1991) 
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The Rural Residency Course 

Mohi Eldin M.A. Magzoub, Mubark M. A. Magzoub and Abdall A. Saeed 
University of Gezira 
Wad Medani, Sudan 

Abstract 

This paper presents the student and graduate experience of the rural residency course 
which aims to improve the performance of doctors in rural hospitals. 
A questionnaire was distributed to the final and penultimate final year students, and to 
graduates working at the Wad Medani Teaching Hospital The response rate was 57%. The 
results have shown that the course was well received by the students and graduates. 
Positive educational experiences were mentioned by both students and graduates. Problems 
facing this course and how these may be resolved are discussed 

Introduction 

Sudan is the largest country in Africa with an area of one million square miles. As one of 
the least developed countries, Sudan exhibits all the features of poor development, such 
as a high birth rate, high infant mortality rate, rapid growth and low gross national product, 
as well as predominance of epidemic and endemic infectious diseases. The majority of the 
population live in rural areas (75%) and 11 % are nomadic. Yet more than 80% of the 
doctors work in big cities, predominantly in the capital city, serving only 14% of the total 
population. 

This problem is mainly due to undergraduate education which is focused primarily on 
practice in big teaching hospitals. These hospitals are relatively well equipped and staffed. 
The health problems dealt with by these hospitals do not represent the commonest 
problems in the community; they are just the tip of the iceberg. A doctor with this 
background, when asked to work in rural areas, finds that he is inadequately prepared to 
perform the tasks that are required of him, let alone be able to contribute to the 
improvement of rural health services. These tasks include supervision of all primary health 
care units in the catchment area. In addition he will be responsible for managing his own 
hospital, including staff, equipment and finance. These duties require administrative and 
managerial skills which he lacks. In most cases the doctor either delegates these 
responsibilities to senior auxiliary staff or becomes frustrated while waiting eagerly to 
finish his compulsory one year rural residency period. The only motivation for him to stay 
is the permission to pursue postgraduate studies. 



Request for reprints should be sent to Mohi Eldin M.A. Magzoub, University of Gezira, Faculty of Medicine, 
P.O.B. 20, Wad Medani, Sudan. 



ERIC 



101 



106 



The Gezira Medical School was established in 1 978 and adopted an innovative curriculum 
based on problem-based, community-oriented and community-based ed u ca t ion. Emphasis 
is on primary health care and rural health. With this bacl^rouiKl the school addresses the 
above human resource problem through a variety of educational activities. One important 
activity is the rural residency which aims to improve die performance of doctors in rural 
hospitals* 

The Rural Residency 

The course is offered to semester 7 students after they have studied all the body systems 
in integrated blocks. This one month course is designed to expose the students to real life 
situations in rural areas and to the nature and responsibilities of the physician Working in 
a rural hospital* 

The course objectives are attained by students observing, assisting, or performing different 
activities in the rural hospital and other health facilities in the area, community* 
In the rural hospital the students observe, assist, or perform different clinical and laboratory 
procedures in a real life situation with the limited resources that are available in the rural 
hospital. These activities are performed under close supervision by the doctor in charge. " 
The students may also live with the doctor or other senior health personnel and work 
closely with them in service administration and management activities. 

The students are also exposed to working situation in other health facilities, including 
health centres and primary health care units. The students are expected to appreciate that 
health services are rendered at different facilities by different health personnel working as 
a team. 

The students carry out a community-based survey addressing a priority health problem or 
health related problem. The purpose is to train the students in the application <?f criteria for 
selecting a priority problem, as well as to design, conduct and analyse a survey. In addition, 
students write a final report in which they describe their experience at the rural hospital* 

The course is preceded by a workshop in which students, staff and government officials 
participate. The workshop discusses course objectives, logistic support and evaluation of 
the course. The students choose their hospital, and only one or two students go to each 
hospital. The university offers a transport and catering allowance. Accommodation is the 
responsibility of the host hospital and of community leaders in rural areas. The course was 
offered for the first time in 1981 . Since then ten rotations have been conducted. 



A questionnaire was distributed to the final and penultimate final year students, who had 
completed the course, and to all Gezira graduates who were working at Wad Medani 
Teaching Hospital; some of the graduates completed their rural residency assignment as 
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doctors. 80 students and graduates responded to the questionnaire, a response rate of 57%.. 
Structured long interviews were conducted with a few students. Also the personal 
experiences of the first author as former student and the second author as course 
coordinator were further sources of information. 

The questionnaire covered organization of the course, attainment of the objectives, 
students 9 perception of the educational impact of the course, and graduates 9 perception of 
the educational impact of the course on their performance at rural hospitals. The response 
to each question was rated from 1 to 5, where 5 represented strong agreement 

Results 

54 (66.7%) of. the respondents were students and 26 (23.3%) were graduates of whom 15 
had completed their rural residency as newly qualified doctors. 63% were males, and 37% 
were females. The combined mean responses and the standard deviation for each question 
are shown below. 

Mean SD 

L Course Organisation 

Enough help from the course coordinator 2.7 1 .3 

Enough help from the doctor at the rural hospital 4.1 1 .0 

Duration of the course was adequate 3.5 . 1.3 

Adequate allowance 1.7 1.2 

n. Course Objectives 

Clarity of objectives 3.6 1 .2 
Relevance of objectives 
Recognition of the role of the doctor 

in the rural hospital 3.5 1 .4 

Recognition of the role of the rural hospital 3.2 1 .2 

Familiarity with the rural hospital environment 3.8 1.1 

m. Educational impact on students 

Learning community interaction 3.7 1 .3 

Learning team work 3.6 1.2 

Mastering skills (manual and communication) 3.8 1 .9 

Learning research skills 3.1 1 .3 

IV. Educational impact on graduates 

Help in the rural hospital as a doctor 3.6 1.3 
Performed less well as a rural doctor if s/he 

did not participate in the niral residency course 3.1. 1.1 
Better performance compared with 

non-Gezira graduates 4.2 0.9 
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Discussion 

There was no significant difference between the responses of students and graduates. The 
above results indicate that this course was well received by the students and graduates who 
responded to the questionnaire. Positive educational impact was mentioned by both 
students and graduates. The reasons behind the interest shown by these students may well 
have been this enhanced opportunity to practice the skills they had learned; many 
respondents mentioned that this had been the first time for them to perform particular skills, 
e.g. abscess drainage, paracentesis. 

Although gender was not an important factor, there had been some difficulties with the 
accommodation of female students in small rural hospitals. All the respondents complained 
about inadequate subsistence allowance. Some of the doctors in the rural hospitals were 
not interested in helping students and considered this task an additional workload. This 
aspect was compounded by the rapid turnover of doctors in rural hospitals. As Sudan is a 
large country with many transport problems, some of the students had to travel from 4 to 
6 days to reach their hospital. This greatly handicapped the students' supervision by 
Faculty staff members. This was obvious from the low rating when the students were asked 
about help provided by the staff coordinator. 

However, this course represents the third phase of the community-based experience, and 
the main educational aim is to give students autonomy and responsibility and to provide 
less support from the staff when compared with the students' two earlier experiences. As 
with other community-based courses, assessment was a problem. Variability of facilities 
at rural hospitals and of the doctors' interest in helping students prevented standardisation 
and create an element of subjectivity. 

Conclusion 

Suggestions to solve these problems include the assignment of students to hospitals where 
Gezira graduates are working, as these graduates are orientated about the course objectives 
and their assessment This would hopefully improve the quality of supervision and 
assessment. Also, students should be encouraged to choose rural hospitals with ready 
access to transport and accommodation facilities conducive to proper supervision and 
follow up. 

It is here important to mention that this study has only evaluated the students' and 
graduates' satisfaction with the course. It did not assess the actual impact of the course on 
the performance of graduates in rural hospital which is the ultimate goal of the course. This 
is an area for further research. 
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The Training of District Medical Officers: A Methodology Tested in 
Senegal* 

* 

The Public Health Research and Training Unit 
Institute of Tropical Medicine 
Antwerp, Belgium 



The training of Senegalese doctors is similar to that of doctors in developed countries. It 
does not prepare them well for the responsibilities of a district medical officer - a role of 
crucial importance to public health services and particularly to the optimal use of scarce 
resources. To compensate for this deficiency in training, the Thiis project has developed 
a programme for teaching district medical officers already in service how to organize 
health delivery systems. This three-month programme consists of a series of modules 
aimed at finding solutions to frequently encountered health service delivery problems, 
followed by a period of on-going training in the form of supervision in the field The 
objective of the course is twofold: to have an immediate impact on the quality of health 
service delivery; andtomotivate the doctor in his work which, untilnow, has only required 
the use of his academic training in the area of clinical practice. This article describes the 
methodology used in the initial three-month stage of this training programme. 

Introduction 

The Thi^s Project 

The objective of the Thifcs project is the improvement of primary health care and its 
supporting referral structures, in the context of a clearly understood programme of 
community participation (Bquipe du projet de Thifcs). The project has been implemented 
by the Institute of Tropical Medicine in Antwerp, under the auspices of Senegal 9 s Ministry 
of Public Health, and is financed by WHO. Activities include the initial training of district 
medical officers, their supervision in the field (which is seen both as in-service training and 
as management assistance), operational and action research in order to adapt national 
public health programmes and better integrate them into existing structures, and, finally, 
assistance in planning at the central Ministry of Public Health level. This article describes 
the objectives and methodology of the initial three-month training period. 

♦This paperwas originally published inHealthPoIicyandPlanning p 4(2),Ui-\56(l9%9^ 
and is here reproduced by kind permission of the editor and the authors. 
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Responsibilities of a Doctor in Charge of Medical '^Circumscription" 

The medical 'circumscription' is the Senegalese equivalent of a medical district (OMS 
Bureau rfgional pour l'Afrique, 1985), which includes a health centre (a small hospital), 
a network of outlying dispensaries called health posts, and the administrative structures of 
these facilities. The medical district coincides with the Senegalese "departcment" (unit of 
local administration). Not too long ago, the responsibilities of the district medical officer 
were still limited to curative treatment, the administration of the services of the health 
centre, the administrative control of the medical district and the management of the budget 
allotted by the government. These responsibilities are currently being redefined in order 
to encompass all the implications of a policy of primary health care. (WHO, 1978) Health 
posts should be responsible for a clearly defined section of the population and provide more 
services than they do at present Apart from curative treatment and vaccinations, they 
should be responsible for patients suffering from tuberculosis, leprosy or malnutrition; 
they should also provide antenatal and well-baby clinics, and offer family planning 
services. Community participation should be encouraged by the health posts by promoting 
non-medical factors that contribute to health - such as environmental sanitation, clean 
water supply and nutrition. 

Referral facilities should be reserved for the treatment of conditions that are too complex 
to be handled within the scope of primary health care, thus leaving primary treatment to 
the health posts. Medical officers need to know, therefore, how to mobilise the necessary 
human, material, financial and technical resources in order to allow the entire system to 
function in such a way. They require a complete overview of the health systems towards 
which their districts should be working. There is a long way to go before this is achieved. 

Inadequacy of Basic Medical Training 

In developing countries - as indeed elsewhere in the world - only very slow progress has 
been made in adapting the medical curriculum to health service delivery needs (WHO, 
1985). Fewer than 5% of all medical schools have community-oriented programmes or 
provide future doctors with training that truly corresponds to the health problems of the 
people for whom they will be caring (FM6p et al., 1984; Katz et al., 1981). All the efforts 
by medical schools throughout the Third World to gain recognition for their degrees in the 
more developed countries raise questions about the resulting brain-drain, the waste of 
resources and the irrelevance of the skills taught to the realities of local facilities (Pathik 
et al., 1980). Only in rare cases have praiseworthy attempts at modifications in the medical 
school programmes been made in an attempt to reduce the emigration of trained doctors 
and to facilitate their reassignment to rural areas (Buri et al., 1980). 

Shortly before the beginning of the project, a brief survey of 28 district medical officers 
in Senegal emphasized their expressed need for supplementary training in management, 
planning, technical supervision, statistics and epidemiology. Are there more shortcomings 
in university training than we know of? 



ERLC 



0:107 



113 



Clinical decision analysis (Wcinstein et al., 1980) is the methodology that would allow the 
cost factor to be taken into account in order to remodel the clinic with the aim of optimising 
the use of scarce resources rather than simply maximising its effectiveness. It is the 
essential component that is still missing in the clinical training of doctors in developing 
countries. Other factors that are also often neglected in medical training include: 

* how to render medical tests and treatment operational; 

* the relationship between the therapist and his patient; this is often not sufficiently 
analysed, whereas prescriptions, which can be standardised, are imagined to be absolutely 
specific to one patient. 

What, then, can be said about the teaching of public health? It most often takes the form 
of a course on the epidemiology of tropical communicable or deficiency diseases based on 
the otherwise appropriate model provided in the manual edited by Benenson, "The control 
of communicable diseases in man" (Benenson, 1 97S). The themes of i nsuffici en t sanitation 
and health education are recurrent in these courses. Although this approach is not entirely 
wrong, it is not very practical: behaviours that are questioned can, in fact, often be justified 
given their conveyed symbolic values, and the environmental constraints. (Obeyesekere, 
1979; Connor et al., 1986) Furthermore, sanitation, which is considered a sine qua non for 
the improvement of general health conditions, is only incidentally the responsibility of the 
health services, especially in urban contexts. 

This dichotomy between clinical or public health training and practice leads to a lot of 
frustration, waste and ineffectiveness. It represents the gap that the Thifes project has tried 
to bridge. 

Objectives of the Training Offered in Thies to Compensate for the 
Shortcomings of University Training 

Imagine a doctor newly assigned to a CM (medical district). He is immediately overwhelmed 
by primary consultations while the patients desert the departments run by the nurses, where 
they are convinced they will only get prescriptions that they cannot afford, and treatment 
by a "doctor' s underling". This lack of confidence is expressed through limited participation 
in preventive health activities, not to mention promotional efforts. Little by little, this 
doctor discovers that financial resources are poorly managed and/or stolen, that the health 
personnel are dissatisfied, and that the administration is both heavy and bound by routine. 
Even in his medical practice, he becomes frustrated by the lack of diagnostic and 
therapeutic tools. 

How can training help this doctor overcome these all too real situations? 

* by restructuring clinical training in accordance with the principles of clinical decision 
analysis (see above); 

* by analysing health service activities: 

The detection of which diseases should receive priority in primary curative medical 
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examinations? 

How should they be handled? 

What should be the content of antenatal care that does not follow blind routine? 

What should be done with a child who is at risk from malnutrition? 

Which priority diseases should referral medical facilities be designed to handle? 

* by describing the organisation of departments responsible for delivering the above 
services, and for the management of resources; 

* by introducing a research component in the management procedures taught; 

* by understanding community participation not simply as a possible source of manpower 
that can be mobilised through the use of appropriate propaganda, but as representing an 
ethical responsibility and a point of reference. 

These are, essentially, the objectives of public health training in Thifes: learn to identify the 
problems inherent in the above areas; find and evaluate solutions to them. 

Teaching Methodology 

Paedagogical Options 

First of all, it should be stated that the course is largely inspired by proven ideas from the 
Master's in Public Health Programme offered by the Institute of Tropical Medicine in 
Antwerp, 'The international course in health development*' (Mercenier, 1973). 

Teaching Based on Problem-Solving 

CM (medical district) doctors passively endure poorly qualified personnel, shortages in 
drugs, and financial deficits largely because they are not able to perceive any solutions to 
these problems. These shortcomings, and the resulting inaction, are not inevitable; the 
pleasure an intellectual draws from his work is largely determined by how much his 
knowledge and training are utilised. But the Third World doctor sees the shortages of 
materials as an insurmountable constraint. He or she rarely treats these problems as being 
worthy of analysis or scientific response, because doctors have never been trained to 
examine them in such a way. 

Furthermore, assiduity is a problem when teaching adults. In situations where coercive 
measures are not used, only the interest of the adult pupils can provide sufficient motivation 
for them to continue attending courses on a regular basis. This can be achieved by 
proposing solutions to the real problems they face every day. 

In order to respond to all of these concerns, a training approach based on systematic 
problem solving seemed to us to be the most appropriate; such training provides a response 
to the experiences of our target audience, highlights the reasons behind their possible 
inactivity, the constraints of teaching adults and, above all, emphasizes our central concern 
for improving the performance of the health services. 
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A Modular Structure 

The problems encountered by a district medical officer can be classified according to the 
level of services, the activity, or the resources being considered. Each of these categories 
includes a number of problems whose solutions are interrelated* To give two examples, the 
problems involved in caring for a woman during pregnancy and labour, while also 
protecting the newborn baby from tetanus, are all related to antenatal visits and the 
improvement of obstetric services; access to cash supply depends on having appropriate 
means and sources of financing as well as a solid system of accounting and control. It is 
the interlocking between groups of solutions related to the same practice that has led us to 
adopt a modular approach in die training we provide. 

How are the Modules Designed? 

The structure of all the modules is more or less the same, as the prime goal is one of problem 
solving. It can be summarised as follows: 

* Guidelines are distributed outlining information to be gathered from the observation of 
health agents at work, the study of documents, and interviews with patients and health 
personnel. These observation guidelines are designed to reveal the dysfunctional aspects 
of the health services, areas for possible improvement and the unique characteristics of the 
services involved. The guidelines were prepared by developing questions about the quality 
of the health services (their effectiveness, efficiency, acceptability, and accessibility) and 
by working out the means of responding to these questions at each facility to be observed. 
These guidelines have proved to be particularly useful since the doctors who used them 
were out of practice in observing, and were no longer particularly struck by the anomalies 
of their daily environment 

* The actual observation is done in small groups so as not to interfere with the activities 
of the health facilities. Following each visit, participants share their observations in a 
plenary meeting. During these meetings, trainers provide the theoretical framework for 
possible solutions: this differs from giving ready-made formulae in that the discussion of 
problems is raised to the theoretical level, alternatives solutions are proposed, and the type 
of research to be done in implementing these solutions is identified. It is sometimes 
necessary to allocate additional time for lectures on fundamental sciences as these can be 
helpful when designing solutions. For example, lectures on risk theory and how it is 
measured are necessary for the organization of antenatal care; descriptive statistics are 
useful when looking at the epidemiology of malnutrition, as are test sensitivity and 
specificity in the organisation of the health centre laboratory, a set chapter on planning in 
the preparation of district planning, and elements of sociology in the family planning 
programme. 

* The next stage consists of writing a health policy document that describes firstly the 
problems related to a given activity (e.g., the high number of patients who abandon 
treatment for leprosy); secondly, the analysis of various solutions (e.g., should the 
campaign against leprosy be integrated into the activities of the dispensaries, or should 
mobile teams be maintained?); and, thirdly, the appropriate choice. 
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* The participant then goes on to develop a strategy which specifies how the chosen 
solution will be implemented (who, when, where and how). Exercises in which participants 
are asked to quantify the elements involved in the solutions makes it possible to adapt the 
chosen solutions to the specific context of each district (e.g., how much would it cost to 
include chloroquine distribution in antenatal consultations in Kedougou?). 

* Finally, in order to assign tasks to the health post nurses, the participant writes up a 
detailed description of the work that is expected of the nurses in the context of the proposed 
programmes. Combined with regular supervision by the doctor, these instructions provide 
an alternative to costly and ineffective training seminars. 

* At the end of each module, the official in charge of the particular national programme 
that is being studied (e.g., tuberculosis, nutrition) is invited to Thifes to present the main 
points of the national policy, and to compare them with the teaching received in Thifcs. 

The Programme 

The content of the various modules is defined by the primary health care policy which 
should provide curative, preventive or promotional solutions to the majority of health 
problems encountered by the population. The health post is the most peripheral health 
facility where a professional health agent can be found, and as such plays a key role in this 
policy as the link between the health services and the community. For this reason, the 
operation of these health posts is at the core of the instruction we offer. 

The prime objective of the health posts is to provide health services that are also oriented 
towards encouraging community participation. These services (Equipe du projet de ilafes) 
are analysed and taught by modules 2-8 (respectively, curative care, care for the chronically 
ill, antenatal care, well-baby clinics, nutrition, family planning, and, finally, community 
participation, which is the underlying paradigm of the entire course). Module 9 deals with 
the extension of health post activities to reach people in outlying villages using community 
health workers and mobile maternal and child health clinics, while module 10 deals with 
the transformation of the (second level) health centre into a true referral facility, through 
the improvement of both its technical performance and efficiency. 

At this stage of instruction, the participants begin to feel discouraged. How can we even 
consider proposing the integration of preventive and promotional activities, when even 
curative care is not properly carried out at the health post? The subsequent modules are 
designed to answer this question. These management modules deal with how to make 
better use of available resources, that is to say, the training and management of personnel 
(module 1 1); finances and cost accounting (module 12); drug management (module 13); 
information systems (module 14); and administration (module IS). 

The very first module is different from the others in that it provides participants with the 
basic theoretical elements to allow them to judge the quality of a health service and the care 
offered, and it establishes a common vocabulary for staff and participants to use. The 
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ABC's of statistics and epidemiology are also presented in this module, because they are 
useful in the process of data collection. 

Module 16 serves as a transition from the initial three-month training period to the 
subsequent period of supervision of the doctor in the field. Having received all the theory 
related to the creation of an integrated health system, doctors then develop a plan for the 
coverage of their district, and an action plan. The coverage plan describes the location of 
future health facilities and the segments of the population that will be covered by them. The 
action plan is a declaration of intent which specifies the ways in which health development 
will be carried out in the district. 

Finally, modules 17 and 1 8 describe, respectively, how the planning process is carried out 
on the national and regional levels. 

An example of a module: Curative treatment at the health post level. 
Let us start off with some hypotheses: 

* Doctors have not been trained to optimise resources but rather to maximise the 
effectiveness of treatment; 

* nurses have not been trained to diagnose and to treat, but nevertheless have the 
responsibilities of a general practitioner; 

* both doctors and nurses devote less effort to solving patients' problems than to applying 
ready-made curative formulae. 

If these hypotheses proved true, what would one find in the health facilities? 

* that less costly, alternative treatments are neglected; 

* that the tests requested contribute little to the final decisions; 

* that treatment is not continuous (e.g., patients stop taking their antibiotics before 
completing the full course of treatment, prescriptions are not collected); 

* that there is a lack of effectiveness (e.g., diseases with high morbidity or mortality for 
which treatment exists, and that are not spontaneously cured, are pooriy treated). 

These are the types of data that the observation outlines given to participants attempt to 
identify. 

What do participants actually see when they observe the treatment of patients, or consult 
medical registers? As far as the diagnoses are concerned, they see: 

* that the dominant symptoms lead to a variety of questions which are not always rational; 

* that stethoscopes are used in unthinking, ritual fashion; 

* that the research for physical signs is relatively rare (sometimes because there is not 
enough time for the examination); 

* that treatment: 

a) is expensive (the systematic use of quinimax or paluject for malaria, the overuse of 
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injections); b) is useless (vitamins, caraphosulphonate of soda); c) always involves 
medication (whereas many patients would benefit from a more psychotherapeudc ap- 
proach); and d) that the choice of medicines, the dosage and the period of treatment advised 
are often fanciful. 

And what do they observe in the area of health service organisation? 

* Without an enclosed space, no therapeutic dialogue between patient and doctor is 
possible in many health posts; only a sheet separates the nurse and his/her client from the 
lines of waiting patients; 

* furthermore, the characteristics of the patients seen by the doctors are identical to those 
of patients seen by the nurses (i.e., like the nurses, the doctor is involved in primary 
consultations). This is a case of both flagrant under-utilisation of the doctor's skills and of 
ineffectiveness, since patients who could truly benefit from seeing the doctor do not 
necessarily receive this service; 

* patients who have been identified and diagnosed as malnourished receive only advice - 
while a milk distribution programme is simultaneously organised in the same health post. 
This shows that integration remains a futile word. 

After agreement is reached on what participants have discovered through their observations, 
the following theoretical elements are taught: the rudiments of systematic decision 
analysis; the characteristics of critical symptoms; decision trees and clinical algorithms; 
formulating empirical strategies; choosing predominant symptoms and standardised 
treatments; sy stemati sing case referrals to the doctor, an annotated drug list; organizing the 
flow of patients; evaluating strategies; dealing with the psychosomatic aspects of treatment 
and the different aspects of empathy and negotiation with the patient; and essential 
information. In one exercise participants prepare instructions for their nurses which are, 
themselves, decisions trees for diagnosis and treatment (e.g., for leucorrhoea, children's 
cough, and the frequently referred to "ache all over" feeling (called *1oy" in Wolof). 

Discussion 

In an evaluation carried out by the Ministry of Health of ten doctors who participated in 
the Thifes course, the relevance of the approach used, the quality of instruction, and the 
usefulness of the c. arse for the regional or district medical officer's job were all rated 
between "good" and "excellent". However, an assessment of the course's relevance to the 
organisation of health services cannot be based simply on a participant satisfaction survey. 
The extent of changes introduced in the operation of the health services has to be examined 
as well. These changes were the subject of another evaluation (Ministers de la santg 
publique du S€ nlgal, 1 987), the results of which, though promising, are too premature for 
us to draw final conclusions. Since we cannot yet analyse project outputs, let us therefore 
examine the unique aspects of this course. 
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Target Population 

The course is devised for district medical officers. They were chosen over other categories 
of health personnel because of the importance of the medical district in the planning and 
management of the health pyramid (OMS Bureau regional pour l'Afrique, 198S). 
Nevertheless, central to the course is the goal of improving the performance of health post 
nurses on whose shoulders lies the main responsibility for the actual implementation of 
primary health care. In this sense, the main concern of the Thifes course is to train health 
team leaders. 

Health policy options (L'6quipe du projet Kasongo, 1981) 

If people do not actually use the health services, both the advice given and the promotional 
health initiatives taken by the personnel of these services cannot possibly be listened to by 
the community members. It is for this reason that the accent of the Thifes course is on the 
rehabilitation of the health services. The policies for achieving ihis goal include the 
development of an integrated health system; the delivery of on-going general treatment 
through primary health care; the participation that is needed from the people in order to 
define with them the ways of promoting health service development and, at a later stage, 
their participation as a means of achieving social dynamism (WHO, 1978). 

Management Policies 

The management techniques that are employed vary according to the health problems to 
which they are applied. Total commitment to the rules of any one school of management 
thought is not appropriate. Let us consider a few examples. In the case of access to curative 
treatment, the approach of the public health service is similar to that of a commercial sales 
area, where the objective is to extend market coverage by creating "selling outlets". On the 
other hand, the attempt to render curative examinations effective requires thinking similar 
to that needed to organise a production unit. A systematic approach is used to deff e the 
interaction and tasks of the various health facilities. A Taylorian approach (Taylor, 19S4) 
is applied to the issues of delegation of authority, and the definition of technical and 
administrative lines of authority. 

The Roles of the Teaching Staff 

The Thifes project has four main functions: in addition to the actual training of district 
medical officers, these include the supervision of the Thi&s-trained doctors in the field, 
operational research, and planning assistance on the Ministry level. These functions are 
mutually supportive. Supervision, for example, gives trainers first-hand knowledge of 
field realities which can make their courses more relevant, allow them to make more 
realistic planning recommendations, and hopefully results in a clearer definition of 
research targets. It is, therefore, preferable that one team is directly responsible for all 
activities, and that excellent communications would be maintained with all sections of the 
Ministry. 
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Teaching Methods 

The information used as subject matter for the course is drawn from the participants 9 own 
experience, which is brought out during discussions and seminars. But the project has also 
taken advantage of the proximity of health services to identify their dysfunctional aspects, 
to learn to observe and, as a result, to supervise. It is a challenge for public health instructors 
wno are not fortunate enough to have their own "testing ground 0 , to teach students not to 
interfere in an activity but to identify errors, to understand the logic of these errors, and to 
select key problems whose solutions may help to move the overall organisation of services 
closer to the development of an integrated health system. 

Training lasts for only three months, because it would be difficult to remove 10 of the 
country *s 40 district medical officers from their posts for a longer period without seriously 
hamperin g the operations of the health services. This constraint requires that the programme 
is condensed and this is ar' lievedby not teaching the fundamental sciences systematically. 

An objection to this methodology then comes to mind: is there not a danger that this type 
of instruction will train managers who tend to be inflexible? It *s the responsibility of the 
supervisors to guard against thL possibility by providing more opportunities for operational 
research within the scope of the management practices of the doctor. The se supervisors, 
therefore, have to have a Master's in Public Health. Investment in such training for the 
district medical officers themselves would not be cost-effective given their approximately 
60% attrition rate and their average 3-4 year term of service. What also distinguishes the 
Thifcs course from a Master's programme is the relative absence of international health 
literature and non-Senegalese case studies, which have been eliminated in order to render 
the course more directly beneficial and to avoid creating a need for access to international 
literature that will be difficult to satisfy in the future. 

Should the teaching role of the project be perpetuated? 

Teaching young medical students how to organise health services is not an easy task, in that 
Ae training has to respond to questions they have not yet had the chance to ask, given their 
lack of experience in the field. Nevertheless, it would be logical for the university to assume 
responsibility for this instruction, even if it were to be programmed as in-service training. 

Under what conditions should this be done? Some members of the teaching staff would 
need to have the field experience of a district medical officer rather than just simply clinical 
or academic experience, or experience in the administration of vertical programmes. 
Instructors would have to continue living the realities of the medical services, albeit on a 
part time basis. It would not be sufficient for them merely to have responsibility for a pilot 
health zone, the logic of which can be so different from the normal heclth service situation, 
both in terms of available resources and concerns. We have already mentioned how useful 
it is to have the four roles of instruction, supervision, research, and assistance, in the 
planning carried out by one, multi-faceted team. The diversity could serve as an 
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appropriate context for such a team, but only if the presence of the appropriate skills could 
allow the university to alter its isolated position. Without such skills at the university level, 
it would be better to maintain the Ministry of Public Health as the institutional base of the 
team. The department of public health would then have to develop a long tci m policy to 
house the team of "public health gcneralists", by hiring qualified associates and by 
modifying its instructional and research objectives. More specifically, this would require 
that the public health department should bring about changes in clinical care and the 
training that is offered in this area at the university to make it more practical. To do this 
would require: 

* the creation of lines of communication between other departments of the medical school 
and the peripheral health facilities in order that both can identify subjects for useful 
research; 

* the introduction of statistical, epidemiological and decision-making tools currently 
lacking in the departments of the medical school; 

* the instruction of medical students in the operational concepts of public health that are 
usefui to gcneralists, as well as the premises of the organisation of the health services. 

The possibility of offering a Master's degree would be useful for future, high level 
officials. This would make sense only if the conditions mentioned above, particularly those 
concerning qualifications, were met, and if career opportunities existed. Certain of the 
above reforms are well under way in the developed countries. They are based on the 
condition that the universities must leave their ivory towers and that the public services 
should introduce research into their operations. Delays in the adoption of these reforms 
would be all the more harmful to developing countries where it is more important than 
elsewhere in the world that resources are managed to produce maximum benefit. 

Note 

The field work was done in Thifcs by Drs. P. Daveloose, A. BS, N.N. Toure Sene and J.- 
P. Unger, under the supervision of Professors P. Mercenier and H. Van Balen, directors of 
the Public Health Research and Training Unit of the Institute of Tropical Medicine, 
Antwerp, Belgium. This paper has been prepared by J.P. Unger. 
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Abstract 

Programme evaluation of six longstanding innovations in community-oriented, problem- 
based medical education was studied at McMaster University, Ben Gurion University, The 
University ofLimburg at Maastricht, Newcastle University, Michigan State University 
andthe University of New Mexico. Three key aspects ofthe evaluation of these programmes 
are addressed First, key questions most frequently asked by programme directors and 
evaluators are described, and selected examples from the six schools are provided 
Second, twelve factors which limit programme evaluations are presented and briefly 
discussed Third, some of the political issues inherent in programme evaluation are 
described. Recommendations and guidelines are offered to programme leaders and 
evaluators who are actively involved in, or contemplating curricular change at health 
institutions. 



Introduction 

In North America, private foundations, such as the W.K. Kellogg, Pew Charitable Trusts, 
Rockefeller, and Robert Wood Johnson, are attempting to move medical education toward 
population and community perspectives (Bruce, 1989; Schroeder, 1990; Rimel, 1989). 
The need for similar efforts is recognised in Africa, Asia, Europe and Latin America. If 
"Health for All by the Year 2000" is to be more than a slogan, each institution concerned 
with the education of future health professionals must act to increase the emphasis of its 
programmes on relevance to the community and to strengthen the partnership between 
communities and responsible institutions. 

Requests for reprints should be sent to Stewart P. Mennin, University of New Mexico, School of Medicine, 
Department of Anatomy, ALbuquerque, NM, 87131-521 1, United States of America. 
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The World Health Organization, the Network of Community-Oriented Educational 
Institutions for Health Sciences and the World Federation for Medical Education are 
actively promoting community-based and problem-based learning as contemporary 
educational methods consistent with these goals (Goon, 1989). Most of the world's more 
recently developed medical schools, and a great majority of the current innovations in 
medical education feature community-oriented (World Health Organization, 1987; Richards 
et al., 1987) and/or problem-based learning (Barrows, 1980). While the vast majority of 
the world's institutions that train future health practitioners continue to use traditional 
educational approaches, there is- a growing interest in the methods and outcomes of 
innovative programmes. 

A handful of these innovative programmes are more than a decade old and have matured 
sufficiently to permit some analysis of the processes of their programme evaluation. 
Informal surveys and interviews were conducted with programme evaluators from 
McMaster University in Canada, Ben Gurion University in Israel, the University of 
Limburg at Maastricht in The Netherlands, Newcastle University in Australia, Michigan 
State University and the University of New Mexico in the United States. Questions 
addressed the structure, context and support of programme evaluation; types of data 
collected and data management; impact of evaluation on the programme; the admissions 
process; and limitations and constraints faced by evaluators. 

Each of these institutions had expressed its educational philosophy in the use of community- 
oriented and/or problem-based learning methods (World Health Organization, 1987; 
Richards et al., 1987; Barrows et al., 1980). More importantly, they had accepted the 
challenge of documenting and demonstrating the merits of their medical education 
programme. It is important to note that traditional measures alone may not assure that the 
innovative features of these programmes are assessed adequately. The evaluation questions 
and methods developed to assess traditional curricula are important, but may not suit 
community-oriented and problem-based curricula because they are unlikely to capture all 
the goals of these new programmes. 

The characteristics that were sought by evaluation studies of recent innovative medical 
curricula have been described (Friedman et al., 1990)*. These studies investigated 
hypotheses of difference in psychosocial and interpersonal skills, continued iife-long 
learning, professional satisfaction, the behaviour of practitioners, educational ach ievement 
and cognitive development, and the nature of the graduates' continuing relationship with 
their parent institution (Friedman et al., 1990). It was also predicted that no differences 
would be found in the passing rates on licensure and certification examinations, likelihood 

* This paper has been reprinted in this Volume of the Annals with the permission of the 
authors and editor. 
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of making major clinical errors, clinical problem solving related to common medical 
problems, and cost efficiency of practitioners (Friedman et al., 1990). Furthermore, it was 
predicted that the educational costs of operating the programmes would not be significantly 
different (Friedman et al., 1990). 

The present manuscript is intended for programme directors and evaluators at institutions 
that are actively involved in, or contemplating change in their curriculum. The authors 
address three key questions about the evaluation of community-oriented and problem- 
based curricula: 1) What questions are most frequently studied by programme evaluators? 
2) What are the factors which limit evaluations? and 3) How can programme leaders and 
evaluators address issues concerning the politics of programme evaluation? 

Key Questions Commonly Aked 

The three types of most commonly asked questions are related to differences, abil ities, and 
impact. 

1. In which ways are students in the new programme different from those in other or 
previous programmes? Are they safe with patients? Have the students been harmed by the 
programme, i.e. will they be able to compete successfully for postgraduate positions and 
future employment? Have the academic staff fulfilled their responsibility to the students 
and to the institution? 

Studies comparing students from innovative medical school curricula (problem-based 
and/or community-oriented) with those from more traditional programmes have reported 
differences in clinical problem solving abilities (Patel et al., 1991). Major differences in 
clinical skills and clinical performance have not been found (Schmidt et al., 1987; 
Kaufman et al.,1989). Students from problem-based tracks tend to perform as well or 
somewhat lower on traditional measures of medical knowledge than their peers in 
conventional curricula (Schmidt et al>, 1987; Kaufman et al., 1989; Goodman et al., 1991 ; 
Hamdy et al., 1991 ; Neufeld et al., 1989). There is some evidence that student attitudes and 
levels of stress are affected by innovative programmes (Moore- West et al., 1989). Finally, 
students from innovative programmes perform as well or better during residency 
(postgraduate) training (Woodward et al., 1981; Woodward et al., 1983; Santos-Gomez et 
al., 1990). 

2. What has been the impact of the innovation on the parent institution, on other health 
education institutions, and on defined populations? The impact of a new programme on the 
parent institution or another institution can be assessed by documenting the spread of 
educational philosophy and methods to other related health education programmes. This 
has occurred at McMaster, Maastricht, Newcastle, and New Mexico. However, the 
literature contains little evidence of systematic study to document the spread of educational 
philosophy and methods from one institution to another. The acquisition or loss of external 
funding for the extension and continuation of innovative programmes (Shelineet al., 1990) 
and increased publicity for the parent institution derived from the new programme provide 
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additional examples of impact 

For McMaster, Maastricht, and Beer Sheva, the major extension of their programmes has 
occurred largely at institutions in other countries. With the exception of Calgary and 
Sherbrooke, other medical schools in Canada were slow to adopt changes but have recently 
begun to entertain major auricular revisions. The same is true for traditional medical 
schools in The Netherlands and Israel. The University of New Mexico and McMaster 
University have influenced other medical schools in the United States to start new 
programmes in medical education. In addition the lessons learned in New Mexico about 
the strategy of parallel tracks as a vehicle for change from within established medical 
schools have been adopted worldwide (Kantrowitz et al., 1987). 

The impact of the programme on a defined population is much more difficult to study 
(Tarlov et al 1 989). Although doctors play an important role in the health of a population, 
the extent to which the well-being of a defined population can be attributed to a programme 
is a complex issue involving many other health care providers, as well as social, 
environmental and political issues. The length of time and the resources required to address 
this question are usually beyond the reach of most evaluation programmes (Naggon, 1 987; 
Gordischer, 1987). 

3. How long can the abilities, skills and attitudes acquired during medical education be 
expected to be evident? In North American medical schools, postgraduate training can last 
as long as undergraduate training. Because of its length and the intense socialisation 
process which occurs, postgraduate training has the potential to erase or cancel out 
intended effects of undergraduate training. Behaviours and abilities can be acquired after 
undergraduate matriculation and contribute confounding variables to the evaluation data, 
thus complicating its analysis (Kassenbaum, 1990). The rapidly changing environment in 
which health care is practised means that the workplace and practice environment 
encountered by new graduates may not be supportive of innovative approaches which were 
part of their training. It can be difficult to ascertain the degree to which educational 
interventions are related to behaviours in practice (Mawardi, 1 979). While most institutions 
are required to gather data about their graduates, attempts to link these data to outcomes 
of innovative programmes can help programme leaders and evaluators to justify the 
resources that are necessary to carry out their evaluation. 

Limitations of Programme Evaluation 

Can the programme be evaluated (Rutman, 1984)? A realistic understanding of the 
potential and actual limitations of evaluations is essential, if programme leaders and 
evaluators hope to assess with accuracy the impact of educational interventions on the 
professional decisions of students and graduates. Several important limiting factors have 
been described. 

1 . Essential factors related to the admissions process which influence programme outcomes 
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are: random selection of students (CanoValleet a!., 1987;LiXue-minetaL, 1987;Lipkin, 
1989); the degree of reliance on academic qualifications (Maddison, 1990; Antonovsky, 
1 976); the extent to which students are selected for their understanding of, and ability to 
fit within the programme (Maddison, 1990; Antonovsky, 1976; Martincz-Burrola et al., 
1985; Brazeau et al., 198S; Ferrier et al., 1978); the age of students upon entry into 
professional training, and the degree to which local communities participate in the 
selection of students (Siega-Sur et al., 1987); and national, state and provincial limits 
specifying eligibility for admission to the institution. 

2. Theavai lability of postgraduate podttoiis can influence the career choice and geographic 
location of graduates from innovative programmes. The larger number of physicians 
competing for relatively few postgraduate training opportunities in Europe (Deliege, 
1987) and the limited positions available in tertiary care centres in Israel (Ellencweiz, 
1983) are two examples. 

3. The availability of expertise in evaluation may be limited. Consultation may be 
available through WHO. Collaborating Centres and organisations such as the Network of 
Community-Oriented Institutions for Health Sciences, and the World Federation of 
Medical Education. 

4. The technical capability to collect and analyse evaluation data or to follow individuals 
longitudinally may be limited. It is not always possible to obtain, store and retrieve routine 
data. Longitudinal studies amass ever increasing data. Some simple, low cost methods for 
more limited evaluations have been described (Mennin et al., 1987). 

5. Financial limitations on programme evaluation are often severe. It is exceptional to 
find educational institutions, such as Maastricht and New Mexico, with personnel and 
resources dedicated to ongoing and longitudinal programme evaluation. 

6. The availability of secondary data sources, such as those used so effectively by 
Woodward and associates(Woodwardetal., 1990), may be limited elsewhere ** Secondary 
data are derived from the trail of records and documents, such as billing patterns, which 
health providers produce as part of their routine activities, or from routine information 
collected by professional or licensing bodies. Two advantages of secondary data are that 
they are more objective than self-report through surveys or interviews and can provide 
substantial information at a comparatively low cost. In the United States, however, there 
are hundreds of different health insurers. This makes large scale studies, which require the 
availability of secondary data, more difficult to conduct. There exist some significant data 
bases in the United States (Erdmannetal., 1986; Jonas etal., 1991), but they have not been 
used to compare graduates of innovative and traditional programmes. Medical schools 
located in countries with centralised health systems and data collection can best use the 
potential information stored in these data bases. Poor record keeping or lack of centralised 
records makes acquisition of secondary data difficult. 

7. The availability of national examinations for licensure provides some comparative 
data. However, these examinations may not test for or provide resolution of differences 

** See paper by Woodward in this Volume of the Annals. 
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introduced by innovative programmes. 

8. Fsw programmes appear to have specific mission statements or sets of goals that lend 
themselves to evaluation. More common are vague statements about producing 
undifferentiated graduates capable of meeting the diverse needs of society. The more 
specific the articulated goals, the easier the programme is to assess. 

9. Geography can be a significant barrier to the assessment of programme outcomes. 
Large distances, poor communication, unreliable postal system, difficult roads, and lack 
of availability of spare parts ft r transport are obstacles to obtaining data about students, 
preceptors and graduates as professionals. 

1 0. Variability of institutional commitment to ongoing programme evaluation can disrupt 
the ability to conduct longitudinal studies. 

1 1 . Unanticipated changes in leadership (institutional or programme) or financial support 
can dramatically affect programme evaluation. 

12. The more complex the programme, the more difficult it is to evaluate. Complexity 
requires lengthier and, therefore, more costly evaluations and makes interpretation of 
results difficult. 

The Politics of Programme Evaluation 

The evaluation of new or innovative education programmes has political as well as 
educational implications (Chelimsky, 1987). Change, desirable or not, threatens the status 
quo. Differences of opinion will always exist about the merits of the philosophy of a 
programme and its implementation. The process of innovation and change in an education 
programme will stimulate both advocates and detractors. Information about the ongoing 
implementation, process and outcomes of the new programme will be used by different 
interest groups as sources of power to support or refute their point of view. The bearer of 
bad news may not be appreciated or listened to. 

Who then are the consumers, the stakeholders, of programme evaluation? They are 
students, academics, other staff, administrators, community people, government, funding 
agencies and, most important, the public. Each group has its r ^ific information needs. 
Students need to be reassured that their programme will not harm them, that they will be 
able to complete it, and be able to compete successfully for postgraduate positions and 
professional advancement. 

Academics and other staff require information to assure them that they are providing 
students with training experiences that are appropriate to licensure and adequate to enable 
students to deal with health problems which they will encounter during and after their 
training. They aiso share concerns about student performance on any national examinations, 
as this would reflect on the status of their institution. Programme planners and developers 
require feedback on (he closeness of fit between the intended and actual programme. 
Frequent "in-flight" corrections in educational programmes need to be based on observations, 
outcomes, and information that are systematically collected and analysed by programme 



123 



129 



evaluators. Funding agencies require reports to assure them that resources are being used 
to meet the goals and original intents for which the funds were granted. Government 
agencies (state, federal, etc.) require information that the programmes plan for, and meet 
public needs and specified goals. The public will want a responsive health system that 
provides access to practitioners who meet their needs. 

Programme evaluation can be used to enhance or diminish credibility. Proponents hope to 
prove and opponents hope to disprove beliefs about new educational programmes. A clear 
and concisely written description of the programme is essential todispel misunderstandings 
and incorrect information obtained through rumors and third party discussions. Descriptive 
reports early in the life of the new programme will help to assure accurate communication. 
Important questions which can arise as a result of early descriptive reports are: How well 
do students perform on traditional measures? What new measures need to be developed 
to enable innovative aspects of the programme to be assessed? What attitudinal differences 
of students, academics, preceptors and employers have been measured and can be 
attributed to the programme? What relationships have been established, enlarged, reduced 
or eliminated as a result of the programme? How has the programme changed over time? 

In summary, the following recommendations and guidelines have particular relevance to 
programme leaders and evaluators at health institutions that are actively involved in, or 
contemplating change of their curriculum. 

a) Whenever possible ask questions that are similar to those posed by evaluators at 
institutions that have undergone comparable changes. These questions are useful in 
comparing results with other programmes and institutions. What are the differences 
between students in the new programme and an appropriate comparison group? Have the 
students been adversely affected by the new programme? Are they safe with patients? 
What has been the impact of the programme on the parent institution, on other institutions, 
on the public? How long can die abilities, skills, and attitudes acquired in the health 
education programme be expected to be evident? b) Limitations imposed by financial, 
technical and geographic factors are important to recognise and to consider in a feasible 
evaluation plan. The admissions process and availability of postgraduate positions affect 
the design and outcome of the evaluation, c) Use secondary data sources whenever 
possible, d) Collect comparative data on performance in national and licensure examinations. 
Accept the possibility that these examinations may not demonstrate or emphasize 
innovative aspects of the new programme, e) Simple studies are faster, cheaper and easier 
to interpret an^ report. They can provide critical information in the initial stages of 
programme change, f) Programmes with vague mission statements and goals are much 
more difficult to evaluate, g) Programme evaluation provides data which can be used to 
support or oppose specific points of view with respect to the internal politics of an 
institution. Evaluators must identify and deal with the needs of each group of stakeholders, 
h) Time is a critical factor in evaluation. Plan evaluations with both short and long-term 
goals in mind, i) Study what others have done and the context in which it was done. 
Coordinate replication studies with other institutions to maximise what can be learned. 
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The collective experience of six community-oriented, problem-based innova 
McMaster University in Canada, at Ben Gurion University in Israel, at the Univ< 
Limburg in The Netherlands, at Newcastle University in Australia, at Michig; 
University and the University of New Mexico in the United States can contribu 
development of a theory in practice for programme evaluation in medical ed 
Continued analysis of the evaluation process will be of value to programmes no^ 
the future. 
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Abstract 

This paper is concerned with evaluation of courses and evaluation of the outcome of 
courses as a method for deciding how such courses may be improved The paper identifies 
the need for asking relevant questions, questions that are relevant for those who are 
concerned with the improvements - the stakeholders. How such questions lead to the 
selection of appropriate methods for data collection, how the information can be analysed 
and how the results should be communicated to the stakeholders concludes the paper. 

Purpose and Scope of Programme Evaluation 

As health sciences educators we make many decisions. We determine what our students 
need to learn, select methods and resources conducive to their learning, and set the basis 
on which they will be certified. To make such choices we need information. We need to 
know where our students will be employed, what they will be expected to do, how effective 
our programmes have been in providing the necessary skills, and what modifications we 
should make in the sequence and/or emphasis we gi>e to the learning of particular skills. 
Everything else being equal, the choices we make on the basis of sound information lead 
to better results than choices made without information (Rotem et al., 1983). 

Programme evaluation is concerned with the systematic gathering and interpretation of 
information about a programme. The information is used to make informed decisions 
about the development and management of a programme. It is here a distinct activity 
aimed to improve rather than to prove. It emphasizes the need to anticipate and solve 
problems that are important, rather than problems that are merely easy to measure 
precisely. 

What should be Evaluated? 

Given the many decisions that need to be made in planning and conducting educational 
programmes, evaluation should be seen as an integral part of our educational activities. 
Evaluative information is likely to be useful if it is timely and relevant to particular 
concerns. Information which is supplied too late or which is not relevant is unlikely to be 
useful for planning and managing programmes. Thus, evaluation should aim to supply the 
best information that can be obtained at the time when decisions are to be made. 

Requests for reprints should be sent to Arie Rothcm, University of New South Wales, School of Medical 
Education, Faculty of Medicine, P.O.B. 1, Kensington, N.S.W. 2033, Australia. 
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The Planning Stage 

When wc plan a new educational programme or when we review an existing programme, 
we need to be clear about the purpose of the programme. We need to understand, for 
example, who will be trained, where they will be employed, what they will be expected to 
do and therefore, what they need to learn. 

Beyond the general analysis of education and trailing requirements, information is needed 
for the detailed planning of the curriculum and all the arrangements for learning. The 
formulation of objectives, for example, should be based on information from practitioners, 
as well as subject matter specialists. The selection and design of learning materials, 
teaching and assessment methods, and organisational setting for learning, require careful 
analysis of what is appropriate and practical. The plan should also be justified from a 
conceptual point of view. 

The use of information in the planning process will improve the chance that our programme 
is effective in meeting the needs of our students, acceptable to the funding and/or 
accrediting authorities, and efficient in the use of scarce resources. 

The Conduct of the Programme 

During the implementation of the programme, information is needed about the 
appropriateness of the plan. It is necessary to verify that assumptions about how it will 
work are correct. For example, was it realisitic to expect that the objectives will be 
achieved?, was the sequence appropriate?, were the students getting sufficient opportunities 
to practise skills and to apply knowledge?, were the required books available in the 
library?, were the students making reasonable progress in their study?, did they seem to 
enjoy their learning experiences? 

It is also important during the implementation process to be sensitive to outcomes which 
are not expected or intended. Unintended outcomes, often referred to as the "hidden 
curriculum", may be positive but they may also be undesirable by-products which should 
be avoided. For example, in the course of learning in the clinical setting, students may 
adopt negative views about other health professions. Also, rather than learn how to learn, 
students might learn how to overcome assessment hurdles with superficial knowledge and 
lack of deep understanding. As these are unintended consequences, they can only be 
identified through sensitive review of the actual process of the programme. 

Outcomes 

In the final stage, evaluation questions need to be asked about the 'products' at the end of 
the programme. Important decisions need to be made about progression of students or 
certification of their competence. At this stage it is also necessary to check again the 
assumptions about the appropriateness of what has been learnt in relation to the requirements 
of the field in which graduates will be employed. In this way, the cycle will be completed; 
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it commenced with questions about the learning needs of the potential students and finishes 
with verification that they have actually obtained the competences they require. 

The significant attribute of this approach, as articulated by Stufflebeam and his colleagues 
(1980), is the opportunity to improve the educational programme throughout the process. 
If the outcomes are not satisfactory it may be necessary to reassess the needs to be 
addressed by the programme or its feasibility, it may be necessary to modify the plan (the 
curriculum and other arrangements) and/or the way it is implemented. For example, we 
may determine that there is a need for additional training for teachers or for change in 
administrative arrangements. 

This interactive approach enables us to effect improvements during the process - we do 
not need to wait for failure before we fine-tune our plan or its implementation. 

As some results can only be appreciated after an interval of time, it is necessary to look at 
outcomes at different time frames to determine whether what has been learnt is actually 
applied in practice and whether it has a positive impact on the delivery of service. 

Steps in Design of Evaluation 
Step 1 - Description of the Programme 

The first step involves clear identification and description of the programme we wish to 
evaluate. The description should include details about the major objectives of the 
programme, the target participants, learning activities and organisational settings. 

Step 2 - Identification of the Stakeholders 

The stakeholders are people or institutions who want to know about the programme and 
whose need for evaluative information we wish to address, for example, a funding agency, 
health authority, community, programme staff, students. 

It is increasingly accepted that evaluation occurs in a pluralistic and political context, 
where different perspectives and aspirations are legitimate and where the potential for 
winning or losing is evident. The existence of many stakeholders with different concerns, 
claims and interests is accepted as an object of the evaluation process rather than as an 
impediment The clarification and resolution of their values and requirements is integral 
to the planning and conduct of the programme. 

Step 3 - Clarifying Areas of Concern 

Each stakeholder may have different and even conflicting concerns. The trainers for 
example, may want to know whether the students are indeed qualified to practice. The 
students, administrators and the community, on the other hand, may be concerned, 
respectively, about the quality of supervision in clinical settings, the use of scarce 
resources, and the attitudes of the health workers towards service in remote areas. 
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It may be necessary to be selective and to draw clear boundaries and priorities in 
consultation with the audiences for whom the evaluation is intended. The issue of timing 
and the potential of the evaluation to support programme development may help to make 
these difficult choices. 

Concerns expressed by one or the other stakeholders may not necessarily be identified as 
problems. Concerns may reflect important values that arc implicit in the programme which 
the stakeholders want to cultivate, and/or such concerns may be critical aspects of the 
programme which need to be monitored regularly. 

Step 4- Formulating Questions 

The formulation of specific questions in relation to each concern or problem is one of the 
most difficult steps in evaluation. Questions need to be broad enough to be meaningful, yet 
specific enough to avoid confusion and different interpretation by different people. 

One of the common pitfalls of evaluation is a tendency to choose questions which are easy 
to answer rather than questions which are important, we should not allow the method to 
dictate the question. 

Step 5 - Formulating Indicators and Criteria 

The next step is to determine what specific evidence - qualitative or quantitative - we wish 
to obtain in order to address the evaluation questions. The evidence could be expressed in 
terms of indicators and criteria. Indicators are variables which reflect the phenomenon that 
is to be evaluated. For example, scores obtained in an examination reflect a certain type 
of learning, extent of participation in community activities reflects interest in community 
life. A criterion is a standard against which judgement may be made. For example, the pass 
rate for an examination may be set at 60%, a satisfactory rate of participation may be 
expressed in terms of 80% of the students attending at least four community meetings per 
month. 

An ideal indicator should be: 



valid actually reflect what it is intended to reflect; 

reliable lend itself to measurement or description with minimum error; 

objective less affected by personal bias; 

sensitive changes in the indicator must reflect changes in the situation or 

phenomenon; 

specific sensitive to the given situation or phenomenon only. 
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Step 8 - Selecting Methods for Obtaining Information 

Given the specific questions and the type of evidence we need to obtain, it is time to 
determine how we can gather the information and *rom what sources. 

The methods of obtaining information may be either quantitative or qualitative. In the past, 
evaluators were often limited in the choice of methods by the expectation that they should 
provide positive proof of cause and effect Unfortunately, hard won knowledge from 
experienced practitioners is often not well documented and not used because it is regarded 
as subjective and not "scientific". 

The design of evaluation should accommodate the complex interaction between variables 
which cannot be readily isolated, controlled, and/or replicated in different situations. 
Qualitative approaches enable detailed description and in-depth understanding of phenomena 
in natural settings (Patton, 1990). In contrasts to reductionist approaches which assume 
that there is an absolute truth, construed vist methods acknowledge subjectivity and help 
to highlight the different perspectives on the same issue. How different people see 
something quite differently may be very instructive but difficult to measure precisely. 

Step 9- Processing the Information 

The type of analysis and summary procedure which is used should be appropriate to the 
evaluation questions) to be studied. Potential weaknesses in the collection of data or in the 
analysis should be mentioned. The categories used for collation should be meaningful, 
internally consistent, and free of logical gaps. Two pitfalls to be avoided in this step of the 
evaluation process are: 

- allowing data collection and analysis to reduce questions to a false implicity. 
Assuming that statistical significance is the same as practical significance, for 
example, finding that only 5% of the students in a training programme complained 
of a particular deficiency does not necessarily mean that the complaint should be 
ignored. 

- using complicated statistics to impress the reader. Some evaluations do not require 
any statistical analysis (Rotem et al., 1983). 

Conclusions in the report of the evaluation should form the basis for modifying the 
programme. The report should be tailored and addressed to bodies or individuals requesting 
the evaluation. Decision-makers should be cautioned about interpreting equivocal findings. 
All conclusions should be defensible (based on logic and appropriate information) and 
defended (reported with possible alternative-explanations, and/or reasons for rejecting a 
conclusion). 

Step 10 - Implementing Evaluation 

If the findings of evaluation are to be accepted and used they must be seen to be relevant 
and credible by the potential users. To this end it is necessaxy to consult the users about the 
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scope and method of evaluation. In the absence of such involvement the users might not 
be committed to the evaluation and hence might be less inclined to incorporate the findings 
into their decision-making. 

The implementation of evaluation requires careful planning and attention to detail. It is 
necessary to determine who will do what, when and how. The resources need to be clearly 
identified. The permission or support of the appropriate authorities should be secured. 
Consideration should be given to opportunities and constraints in implementation, and 
arrangements should be made for contingencies for dealing with potential difficulties. A 
time table should be prepared. 

For an external evaluation it is desirable to draw up an explicit contract with the evaluator. 
The terms of reference should specify the expected role of the evaluator, the framework 
for consultation, the timing for each step, and the expected outcome of the evaluation. 

A truly, responsive evaluation is similar to the work of the detective attempting to unveil 
a mystery. New leads become apparent in the course of the investigation that raise new 
questions and require changes in the methods of investigation (Smith et aL, 1990). Thus, 
far from being static, evaluation is dynamic, cyclical, and a never ending process (Shadish 
etal., 1991). 

Conclusion 

This paper promotes the use of programme evaluation as a tool by planners and managers 
of programmes. The mystification of the evaluation process as a highly technical operation 
should not discourage us from its application. Preoccupation with scientific purity and 
technical elegance should not direct evaluation to problems that are easy to measure, rather 
than problems which are important to solve. Evaluation and research have frequently been 
applied to trivial questions and have produced data under conditions that are not 
necessarily representative and often too late to incorporate in decision-making. 

The advantages of using the decision-making model for educational evaluation include: 

- provision of a systematic approach to evaluation; 

- provision of appropriate information to diverse groups at the time they need it; 

- wider and increased use of information for decision-making; 

- empowerment of stakeholders to participate in all stages of the evaluation; and 

- awareness of the developmental nature of educational evaluation (Weiss, 1983). 

The approach suggested in this paper should have particular appeal to those who are 
concerned with the use of the findings. The high level of participation of the stakeholders 
in forming the questions for the evaluation and in determining what evidence will be 
admissible for decision-making and the methods for obtaining it, can be expected to 
contribute towards a higher level of ownership of the findings and thus commitment for 
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action. This is a learning process which underpins any improvement in programme 
development and 'management. 
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Abstract 

It is argued that curriculum evaluation should not be based solely on given goals or 
programme objectives as this would be unlikely to engender any really significant change 
and even then only within a prescribed framework. Dynamic evaluation for change must 
be able to consider significant issues and contingencies that are part of the changing health 
care environment. 

The dynamic model in the present paper incorporates un appraisal of the external, as well 
as internal context of health care practice. The external context is explored for changes in 
social, economic and professional expectations, changing technologies, international 
relations and salary structures. The internal context is reviewed with respect to the 
changing abilities of students, needs of academic staff and the changing roles of institutions 
of higher education. 

The purpose of this paper is to present a model for evaluation of health professional 
curricula which is sensitive to changing health care needs, policies and systems and to the 
characteristics of educational institutions, providers and students. 
It is argued that movement towards the goal of Health for All is dependent on a change in 
the knowledge and skills of health practitioners, and that this will only be achieved if 
curriculum evaluation is based on an ecological perspective of health, an holistic view of 
human beings and a dynamic, interactional orientation to the process of education. 
Accordingly, the first part of the paper explores the evolving holistic or ecological 
approach to health in Australia, and the contradictions between the rhetoric and the 
current reality. Further, it argues that the desired change in health professional practice 
would be assisted by the adoption of an appropriate framework for evaluating and 

* Based on a paper presented at the Annual Conference of the Australasian and New 
Zealand Association for Medical Education, "Evaluation for Change", July 7-10 1991, St 
Hilda's College, The University of Melbourne, Australia. 

Requests for reprints should be sent to Jenny Graham, The University of Newcastle, Faculty of Health 
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changing health professional curricula, which recofri&s changing assumptions about 
health and health care and which therefore, would lead to better preparation of health 
professionals, regarding their capacity to initiate and support change. 
The second part of the paper provides a brief overview of approaches to curriculum 
evaluation and their deficits. Arguments for a new framework for curriculum evaluation 
are advanced 

The third and fined part presents a dynamic model for curriculum evaluation, including a 
detailed illustration of the types of factors (in external and internal contexts) which require 
consideration. 

An Evolving Holistic orEcological Approach to Health and Implications for the Education 
of Health Professionals 

The dominant concern in society today is quality of life, and how to maintain or enhance 
it. Across the globe governments, health authorities, researchers and individual people are 
struggling with the challenge of ecologically sustainable development 

In response to a recent Australian Commonwealth government discussion paper on 
ecologically sustainable development (1990) and the National Health and Medical 
Research Council report (1991), endorsing the 1989 European Charter on Environment 
and Health, asserted inter alia that the health of individuals and communities should take 
clear precedence over considerations of economy and trade; that there is a responsibility 
to demonstrate that all aspectsofsocio-economicdevelopment(ncw policies, technologies, 
etc.) are not harmful in their impact on the environment and health; and that development 
assistance should promote sustainable development and the safeguarding and improvement 
of human health as one of its integral components. Commensurately, they recommended 
that the emphasis in society should be changed from material consumption to quality of 
life. However, whether the goal of sustainability in health can actually be achieved has yet 
to be seen. 

In Australia, despite the initiation of a comprehensive review of the health sector (Macklin, 
1990), it would seem that we are still orientated towards a dependence and illness model 
of health, rather than towards a focus on self-sufficiency and quality of life. From a global 
perspective, as Timberlake and Thomas (1990) point out, governments, UNICEF and 
WHO are planning and delivering health care outside communities, rather than assisting 
communities to take responsibility. This approach is harmful to sustainable development 
because it thwarts any attempts by communities to determine their own health needs and 
ways to meet those needs. 

The extent of the problem is reflected in large measure in the theme of the Third World 
Conference on Health Promotion in Sundsvall, Sweden, treating Supportive Environ- 
ments". As the conference working papers (1991) point out, while the concept of 
supportive environments is relatively underdeveloped and has been difficult to translate 
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into operational terms, it is imperative that health promotion strategies recognise the role 
of supportive environments in providing the prerequisites for health development and 
equity in health. 

The degree of the problem is also reflected in a recent issue of the Australian Journal of 
Public Health , where Nutbeam (1991) argues that, "because the reward system in the 
health services is largely based on achievement related to measures of throughput and cost 
containment, rather than to health outcomes, service managers inevitably have a narrow, 
short-term focus, rather than a public health perspective directed towards optimal community 
health", and "that there are no tangible rewards for improving community health and no 
sensible short-term indicators available to measure progress towards this goal". 
Consequently, "public health in general and health promotion in particular are always in 
danger of marginalisation". 

In summary then, the rhetoric is sound, but its translation into practice is wanting. As 
Graham and Honari (1991) suggest, the assertions of Boyden (1980) are still applicable 
today: "Human beings do not all respond alike physically, emotionally and socially, and 
we must take account of what they feel comfortable with and/or fulfilled by (the intangibles 
~s well as the tangibles) if we are to facilitate health. The insidious and progressive 
dehumanisation, of which so many people complain in modern society, must now be taken 
seriously. It is time that we made a deliberate effort to bring the intangibles out into the 
open, to study them systematically, to improve our language for discussing them and, on 
the basis of the various approaches which are available to us, to see whether we can reach 
a degree of consensus, with respect to their relative importance for the quality of life". 

What is required is a fundamental shift towards a real, holistic view of people and their 
health; and health professional educators have a primary responsibility to assist in effecting 
the necessary changes. 

This idea, of course, is not new. The WHO document "Learning together to work together 
for health" (1988) suggests that "basic education programmes in nursing, medicine, 
medicosocial work and other health professions should aim at producing generalist 
practitioners who are able to see each patient or client both as a whole person and as part 
of society." The Report is critical of "the division, even atomisation of the typical, single- 
profession health care curriculum, which orientates students towards specialisation and 
away from general and multi-professional practice", and which "neglects the contribution 
which other health professions and non-health sectors could make". The report also points 
to "multi-professional education as a means of remedying these effects"; and that 
multiprofessional research at the undergraduate level may be used as a learning tool. 

Similarly, at the Australian Health Ministers Conference in 1988 (Health Targets and 
Implementation Committee, 1988), the need to develop a plan for the holistic training of 
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health care practitioners was discussed together with the need to promote education for 
Primary Health Care and for such education to focus on intersectoral action at the local 
level. Hie Conference acknowledged that the health system can only deal with some of the 
factors which influence health status and that policies and strategies in other sectors 
including welfare, finance, housing, agriculture, and industry, have important health 
implications. 

Of course, many within the health professions concerned with education echo similar 
sentiments, but the problem, as indicated by WHO (1988), is that practice encourages a 
much more blinkered view. We still live in a world (Graham, 1990) where efficiency and 
cost containment dominate concern for quality of life, where universities reward research 
rather than teaching excellence, where the non-medical health professions, despite 
protestations to the contrary, are still seeking to emulate medicine, and where intra- and 
inter-professional rivalry impede change in curricula. 

Nevertheless, we suggest that change is possible if the whole process of curriculum design 
and evaluation is predicated by a human ecology driven, holistic view of humankind. This 
emphasizes: 

* the health care needs of people who are individuals with respect to their physical, social 
and spiritual well being, their cultural identities, and their attitudes and values; 

* the necessity to view individual health and health care within the context of community, 
national and global health needs; 

* the dynamic relationship between health and all aspects of the environment (including 
physical, biological, technological, socio-cultural, economic and political); 

* the importance of the concept of sustainability in the orientation, development and 
management of health services; 

* the importance for all health care professionals to empower and collaborate with 
individuals and communities in solving problems of health care. 

This, of course, requires a shift of perception similar to that which occurred in physics in 
the 1920s, when the exploration of atomic and sub-atomic worlds led to a movement away 
from the mechanistic conceptions of Descartes and Newton to an holistic view (Capra, 
1982). It means a coming to grips with the inadequacy of traditional concepts and adjusting 
our language and our whole way of thinking. It means not merely an intellectual exercise, 
butdealingwithaprofoundemotional,even existential crisis. Otherwise, health practitioners 
and health professional educators will perpetrate and probably exacerbate, albeit unwittingly, 
inequities in health. 

To be fair, we must acknowledge that the beginnings of the shift - from mechanistic to 
holistic conception of realty - ait already visible. However, as time is not on our side, we 
need to do more to translate the rhetoric into practice, lest we face Health for a Few rather 
than Health for All by the year 2000. 
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Wc would argue that many evaluations of curricula are carried out in good faith, but can 
be laborious, time consuming and insensitive to change. Our focus is on facilitating change 
by providing an appropriate dynamic framework for reflective evaluation. 

A Dynamic Framework for Evaluation 

The literature suggests that there is no consensus on what constitutes curriculum evaluation 
(Westbury, 1970), and that most emphasis has been given to the strategies of the 
evaluator* s task, rather than to theories for interpreting the phenomena of higher education. 

The grandiose vision of the contributions that evaluators could make to educational 
practice is now less emphasized. With the passage of time there has developed a greater 
acceptance of the shift away from the use of a few psychometric measures to an approach 
which incorporates a broader range of procedures and techniques. There has been general 
disillusionment (Cronbach, 1962/63 and Atkin, 1967/68), with the Tylerian (Tyler, 1949) 
pre-ordinate evaluation model. This prompted Stake (1967) to propose a broadening of 
evaluation. Since that time others have proposed such schemes as illuminative evaluation 
(Parlett and Hamilton, 1972), goal-free evaluation (Scriven, 1972) value-free evaluation 
(House, 1974), holistic evaluation (MacDonald 1971, 1976), responsive evaluation 
(Stake, 1973, 1975), and evaluation in naturalistic settings (Smolitz, 1975). These 
represent a significant change in thinking about curriculum evaluation and are in stark 
contrast to the traditional focus on using measurements of individual pupil personality, 
behaviour or other attributes to explain the extent to which explicit curriculum objectives 
have been met or to demonstrate the merits of certain innovations. 

In higher education it is still commonly accepted that course evaluations should focus 
primarily on such procedures as collecting perceptions about the course from colleagues, 
"educational experts" and students, through open-ended and structured interviews, structured 
group discussions and questionnaires (Ramsden and Dodds, 1989). The focus is on what 
has happened in the course and the view is, at best, from the course looking outwards 
(exemplified in such questions as: "Will the content, skills and abilities be relevant to 
professional practice?", rather than looking inwards, that is, putting effort into identifying 
clearly the current needs of professionals and then considering how these needs might be 
met by a course. It is interesting to ponder why an emphasis on context in evaluation, as 
outlined by such researchers as Stake (1967, 1973, 1975) and Parlett and Hamilton 1972) 
in the 1 970s, has not become a prominent part of the evaluation culture in higher education . 
In many cases data have been collected before the purposes of an evaluation have been 
determined; and, after an evaluation, course re-accreditation documents have reflected 
only minor modifications. 

In an evaluation model for change, however, the goals on which the initial curriculum was 
based must be able to be changed. 
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GIVEN GOALS > NEW CURRICULUM GOALS 

Any curriculum evaluation which is only based on given goals will inevitably not change 
to any great degree 2nd then only within the prescribed curriculum framework. This is a 
far cry from the dynamic evaluation which is required, if we are to accommodate the 
significant issues and contingencies which are part of the changing health care environ- 
ment. 

Outline of a Dynamic Curriculum Evaluation Model 

Our dynamic evaluation model incorporates an appraisal of the external and internal 
context of health care practice. The external context is explored in relation to changing 
social, economic and professional expectations, changing technology, internationalisation 
and award restructuring. The internal context is reviewed with respect to the changing 
abilities of students, the needs of academic staff and the changing roles of higher education 
institutions. 

As revealed in the model, the external and internal environments of a course for health 
professionals pose a myriad of questions which can impinge on a number of curriculum 
perspectives: 

* course content ... the disciplinary and interdisciplinary concepts, skills, abilities and 
attitudes; 

* course delivery ... use of lectures, tutorials, workplace learning, independent learning, 
peer assessment, oral work, computer managed learning, facsimile, teleconference; 

* course assessment ... empowerment in learning through self-assessment, use of a variety 
of assessment strategies such as oral, public presentations, clinical diagnosis and cross- 
disciplinary team work on problem-solving; 

* other related issues ... staff development, resource issues. 

The perspectives of the external and internal environment (Appendix) provide a checklist 
for identifying foci for curriculum change. It is important to note, however, that the entries 
in the checklist are intended to serve as illustrative examples, rather than as universally 
applicable absolutes. Equally, it is important to point out that more change can be generated 
through this model than can be accommodated realistically. It is essential, therefore, to 
target change and to develop priorities judiciously . It should also be stressed that this model 
will inevitably indicate the same or similar needs for change at different points in the same 
checklist This may ipso facto assist in deciding on a sequence of priority in the change 
process. 
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A Framework for Course Evaluation 




Current Course 


Context Checklist 


New Course 


Goals 


External Context 


New Goals 


Course Content 


- changing societal expectations 

- changing professional expectations . 

- changing needs for health care 

- changing uses of technology 

- Funding • 


• 

New Course 
Content 


Course Delivery 


- Internationalisation 

- (Global policy) 

- Overseas community 

- National policy 

- Resources 

Internal Context 


• 

New Course 
Delivery 


Course 
Assessment 


* Student 
-Staff 

* Institution 


New Course 
Assessment 



Conclusion 



The need for an ecological approach to health is receiving worldwide recognition. 
Commensurately , we have argued that there is a need to adopt a dynamic, holistic approach 
to education of health professionals and, specifically in this paper, to curriculum evaluation 
and change. 

In our experience, the global collective reflections of a group of health care educators, 
systematically organised, can yield positive changes to curricula far more efficiently than 
the purposeless or narrow focused data collection activities that often accompany the 
evaluation process. It is acknowledged that some judicious data collection activities could 
provide a useful adjunct to curriculum evaluation, but it is also argued that data are often 
collected in a mindless vacuum and at great expense. The mark of quality is the extent to 
which education prepares practitioners to improve future health care practice; and the 
holistic approach to curriculum evaluation, as illustrated by the present model, is deemed 
essential to the process of engendering truly worthwhile change. 
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APPENDIX 



Checklist for Identifying Foci for Curriculum Change: 
Some Illustrative Examples 

External Context 



1. Changing professional expectations 



Professional career structures 
Training guarantee 
Staff development 
Research 
Constraints 



Are professional objectives spelled out 
in terms of such things as: 

. to communicate orally and in writing 

with different communities; 
. to engage in professional development 

activities; 

. to reflect on the needs of the profession 
in relation to other health professions; 

. to identify community perceptions; 

. to establish links with other health 
professionals; 

. to use appropriate research skills? 



What are the implications for course 
content/process? 

- Are adequate communications skills 
being developed? 

* Are there interdisciplinary units? 

- Are students learning to reflect? 

- Are problem-solving skills being 
developed? 

What are the implications for course 
delivery? 

* Should there be more experiential/ 
workplace learning? 

What are the implications for course 
assessment? 

- Have oral presentation and pamphlet 
design tasks been included? 

* Have tasks on investigative techniques 
been included? 

- What problem-solving and value clarifi- 
cation tasks have been included? 



2. Changing uses of technology 

Educational technology: 

- Computers 

- Telephone/Facsimile 

- CD ROM 

- Interactive Video Disc 



What are the implications for course 
content/process? 

* Which skills should be developed? 



What concepts should be developed? 

What ethical considerations should be 
incorporated? 
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Health care technology: What are the implications for course 

- Drugs delivery? 
* Equipment 

- Could/should teleconferencing be used? 

• Could/should self-paced learning be 
implemented? 

* Could/should staff and students have access 
to hospital as well as university information 
networks? What limitations should be 
placed on access? 

What are the implications for course 
assessment? 



3. Funding 

- Constraints on new course development 

- Course innovation 

- Programme attrition 

- Attraction of outside funds 

- Over-enrolment 



* Could/should educational technology be 
used to encourage self-paced assessment? 

What are the implications for course 
content/process? 

- What are the key objectives? 

- Which skills are essential/non-essential? 

- Which concepts need to be learned, dis- 
regarded? 

- Which attitudes need to be developed, 
ignored? 

What are the implications for course 
delivery? 

- How cai classroom contact be reduced? 

- Can more time be spent in the workplace? 

- How can the above be achieved? 

What are the implications for course assess- 
ment? 



- Can more self-assessment be used? 

- Can more peer-assessment be used? 

- Can assessment be made more efficient and 
effective? 



Other implications? 

- What sources are available for infrastructure 
funding? How can they be tapped? 

- Should students pay for access to learning 
resources? If so, which ones? 

- Can course administration be streamlined to 
conserve resources for teaching/learning 
purposes? 
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4* Intel-nationalisation 



• Internationally agreed framework of 
philosophy/attitudes, e.g. 

"Declaration of Alma Ata" 
(WHO, 1978) 

"Global Strategy for Health for All by the 
Year 2000" (WHO, 1981) 

"Ottawa Charter for Health Promotion" 
(Health and Welfare Canada, 1986) 

"Learning Together to Work Together for 
Health" (WHO, 1988) 



"European Charter on Environment and 
Health" (WHO, 1989) 



What art the implications for course 
content/process? 

- Is there a role for comparative studies? 

- Should there be exploration of attitude 
clarification in the programme? 

• Knowledge of repents? 

- Applications of knowledge? 

What are the implications for course 
delivery? 

- What staff-student exchanges should occur? 

- Should there be work experience program- 
mes abroad? 

- Should there be international teleconference, 

research link-ups. 

What art the implications for course assess- 
ment? 

- Assessment of overseas work placement? 

- Assessment of attitudes versus knowledge 
and skills? 

- Contributions to an international data bank? 



5. Overseas students 



- e.g. Code of ethical practice in the pro- 
vision of full-fee courses to overseas 
students by Australian Higher Education 
Institutions (1988) 



- Legislation 



What are the implications for course 
content/process? 

- Are contextually specific terms clearly 
defined? 

- Are comparative cross-cultural issues 
addressed? 

What are the implications for course 
delivery? 

- Are appropriate language and bridging pro- 
grammes provided? 

- Are students orientated to particular learning 
culture/expectations? 

What are the implications for course assess- 
ment? 

- Are specific criteria for assessment clearly 
stated? 

- Is progress carefully monitored? 
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Other Implications? 



6. Community 



- People 

- Industry 

- Social* physical and biological environment 

- Other resources 



7. Government policy and guidelines 



- Articulation of courses (e.g. links between 
university/technical and further education) 

• Equity 

- Staff appraisal 

- Performance indicators/ 
assessment of teaching 

- Research (e.g. policies for ethical clearance; 
targeted government funding) 

- Training guarantees 

- Professional career structures 

- Legal specification/limitation of professional 
responsibility 

- Labour force targets (numbers within spe- 
cialisations) and employment assessment 
opportunities 

- Multi-skilling 



- Are staff development needs addressed with 
respect to managing the learning of overseas 
students? 

What are the Implications for course 
content/procepj^ 

- What skills needs to be developed in 
communicating with the community? 

- What axe the community's perceptions? 

What are the implications for course 
delivery? 

- Should the course have special reference to 
the local community? 

- How can community resources best be used? 

What are the implications of course assess- 
ment? 

- Projects assisting in community empower- 
ment or self-help? e.g. 

. Preparation of community pamphlets? 
. Magazine articles? 
. Talks to the community? 
. Community surveys, cooperatively 
designed with consumers? 

What are the implications for course 
content/process? 

- Critical review skills? 

- Knowledge of reports? 

- Applications of knowledge? 

- Articulation of objectives? 

- Knowledge of health and safety? 

- Objectives related to changes in professional 
career structures? 

What are the Implications for course 
delivery? 

- Multiprofessional learning? 

- Application of health and safety principles? 

- Process for ethical clearance for student 
research? 

- Opportunities for cooperative education 
(e.g. university in conjunctions with industry) 
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- Secondary school 

- Accrediting agencies 

- Copyright acts 

- Health and safety legislation 

• Higher education contribution scheme 

- Reports e.g. 

"Looking Forward to Better Health" 
(Better Health Commissions, 1986) 
"Health for All Australians" 
(Health Targets and Implementation 
Committee, 1988) 

The National Health Strategy: Setting the 
Agenda for Change" (Macklin, 1990) 



What are the implications for course assess- 
ment? 

- Critial review of reports? 

- Surveys of knowledge? 

- Applications of principles? 

- Generic versus specialist knowledge and 
skills? 

Other implications? 

• Appraisal of staff practice? 

- Focused staff development? 

- Negotiation with other educational bodies? 

- Provision of in-service programmes? 

- Review of student entry groups? 
Access for disadvantaged groups? 
Recognition of prior learning? 

- Transfer of credit between university and 
the other courses? 



Internal Context 



{.Students 



- Percentage of students living locally and 
at a distance 

- Different academic backgrounds 

- Different language skills 

- Different value systems 

- Different experiences 



What are the implications for course 
content/process? 

- Incorporation of academic with language 
skills? 

- Evaluation of individual differences? 

- Clarification of personal values and goals? 



What are the implications for course 
delivery? 

- Integration of academic and language skill 
exercises? 

- Experiences of alternative lifestyles? 

What are the consequences for course 
assessment? 

- Assessment of academic skills within two 
weeks of starting the course? 

- Self-assessment? 

Other implications? 

- What are appropriate student entrance 
policies? 
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2. Lecturers 



- Lack of training for their academic role 

- Each with a unique contribution to make 

- Each with different skill levels in terms 
of knowledge of discipline and ability to 
communitcate it 

- Problems engaging appropriate staff in 
some areas 

- Varied ability to contribute to research 

- Increasing workloads in terms of hours 
and students 

- Lecturers have a discipline orientation 

- Lecturers' value systems, such as academic 
freedom 



Provision of professional development to 
extend skills in: 

* Negotiation 

* Curriculum design 

* Programme development 

* Course delivery 

* Assessment 

- Evaluation 

- Interrelating with other health practitioners/ 
disciplines 

- Time management 

Provision of guidelines on course/programme 
development 

Provision of opportunities for professional 
development in existing and new areas, 
through: 

- Time for research, course evaluation 

- Secondment 

- Formal study leave 

- Varying roles and responsibilities 

Recognition of commitment to teaching and 
programme development 

- Recruitment criteria 

- Promotion criteria 

- Conference leave and support 



3. Institution 



- Other courses available 

- Facilities 

* Equipment 

- Other expertise 

- Structures 

- Financial/academic/political priorities 



WI.«i are the implications for course 
content/process? 

- Opportunities for interprofessional studies? 

What are the implications for course 
delivery? 

- Facilities available for different delivery 
modes? 

- Equipment available for teleconferencing? 

- Need to negotiate for more computers? 

- Link up with other institutions? 

- Implications for course evaluation? 

- Assistance available for assessment 
procedures? 

- Can assessment be self-paced? 
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Other implications? 

- Appropriateness of course policies? 

- Provision for advanced standing? 
• Recognition of prior learning? 

- Appropriate change strategies? 
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A Multidimensional Approach to Evaluating a Changing Curriculum 

R. Bandaranyake, P, Craig and R, Wagner 
University of New South Wales 
Kensington, Australia 



This paper discusses experiences with a variety of methods which were used to evaluate 
a changing medical curriculum. While evaluators may be guided by these experiences it 
is emphasized that no single "evaluation template** fits all curricula; the peculiarities of 
each situation determine the optimal evaluation strategy. 

Introduction 

In 1988 the medical curriculum of the University of New South Wales in Australia 
underwent a significant change. It reverted to a six-year curriculum from a five-year 
course, which in turn had replaced a six-year curriculum established at the inception of the 
medical school. The recent change was made primarily to give students more time to reflect 
on their learning, to develop their skills in applying their learning in the basic sciences to 
clinical and community problem solving, and to have more exposure to clinical experiences* 
A specific condition was that there should be no net increase in the content of the 
curriculum, except in certain identified areas which were thought to be of current 
importance. 

Concurrently with the change, the Curriculum Committee of the Faculty of Medicine 
instituted a sub-committee, the Course Evaluation Committee (CEC), and appointed one 
of the authors (RB) as its Chairman. 

The specific terms of reference of the CEC were: 

1 ) to determine the extent to which the objectives of the new curriculum were met; 2) to 
compare each year of the new curriculum with the corresponding year of the old 
curriculum; and 3) to determine whether the new curriculum prepared students better to 
apply the knowledge they gained from each year of the course, to the situations they 
encountered in the succeeding years and in future practice. 

In addition to its Chairman, the CEC consists of the Dean of the Faculty of Medicine, the 
Chairman of its Curriculum Committee and the Chairman of a relevant Year Committee. 
It reports directly to the Curriculum Committee on which the Chairman of CEC sits. In 
addition, the latter is invited to meetings of the relevant Year Committee where CEC 

Requests for reprints should be s?nt to R. Bandaranayake, University of New South Wales, School of Medical 
Education, Box 1, Kensington, N.S.W. 2033, Australia. 
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reports are discussed. The Year Committees are also sub-committees of the Curriculum 
Committee. 

The CEC focused its evaluation activities mainly on the objectives of the change. It 
evaluated: 1) each course within a given year as a whole, rather than its detailed content 
or the effectiveness of individual teachers within it; 2) the links among courses at 
horizontal and vertical levels; 3) the relative emphases on topics within the courses in 
relation to priority health needs; and 4. the total impact of the courses within a year on the 
students in that year. Detailed analysis of course content and teacher evaluation were left 
to each department. Most departments already conduct teacher and subject evaluation; 
others have sought the help of the CEC in setting up such evaluation. 

The evaluation plan established by the CEC incorporated input, process and product 
evaluation. Product evaluation will be undertaken when the last of the five-year, and the 
first group of the six-year students respectively commence their intern training. 
This report deals mainly with process evaluation conducted over the last four years. 

Methods 

1. Staff and Student Survey 

At the outset the CEC decided to evaluate each year of the new curriculum, which was 
being phased in on a j early basis, only on the first occasion when it was introduced. This 
inevitable resulted in the same cohort of six-year students responding to the evaluation 
instruments each year. As the evaluation was partly a comparison with the old curriculum, 
it was necessary to evaluate the corresponding years of the old curriculum as well 
simultaneously. The project started too late to evaluate the first year of the old curriculum 
as it had already been replaced by the first year of the new curriculum. However, evaluation 
data gathered by the individual departments during the previous year were available to the 
CEC. As for the remaining four years of the old curriculum, the CEC was able to conduct 
its own evaluation as these years were still in the process of being phased out. Rather than 
the same cohort of five-year students responding to the evaluation instrument for all four 
years, responses were sought from different cohorts for each year over the first three years 
of the project. 

A modified Delphi technique (Delbecq et al., 1975) was used to seek perceptions of staff 
and students for each year of the cv riculum. About midway through a particular course, 
individual interviews were conducted with course coordinators and corresponding heads 
of departments. Their perceptions about the strengths and weaknesses of the course, and 
their concerns and problems encountered were determined. Approximately one-third to 
one-half of the students in that year were then interviewed in groups of 10-15. At these 
interviews student opinion was sought on staff perceptions. In addition, students were 
encouraged to contribute their own perceptions. The information gathered from staff and 
students was used to develop a student questionnaire, addressing specific issues for the 
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particular year. This questionnaire was distributed to the relevant staff for comment and 
amendment (if necessary) before being administered to students, including those who 
failed, as they re-enrolled in the following year. Student responses were collated and 
analysed, and a preliminary report was sent to the Dean, Chairmen of the Curriculum and 
corresponding Year Committees, and the heads of relevant departments. The latter 
received only data pertaining to their own courses, whereas the others received data 
pertaining to all courses in that year. Course coordinators were asked to prepare evaluation 
reports on each of their courses from the student feedback provided by CEC, and from the 
data from "in-house" evaluations carried out by the department. This report included, in 
addition to the results of evaluation, the course objectives, examination results, innovative 
approaches, and changes made or recommended. 

A similar process was followed for the corresponding year of the old curriculum. A draft 
report for the year was prepared by the CEC, summarising the perceived strengths and 
weaknesses of each course from the staff and student points of view, comparative data for 
old and new curricula and changes made as a result of the evaluation. This report was again 
sent to staff for comment before being finalised and distributed to relevant authorities for 
discussion at a Year Committee Meeting. 

2. Comparison of Performance on "Marker Questions" 

Old and new curricula were also compared on the basis of student performance on common 
("marker") questions, essay and multiple choice questions (MCQ), set in end-of-course 
examinations. For obvious reasons, only a small number of questions could be repeated 
each year. This comparison was abandoned in the later years of the course due to lack of 
equivalence between old and new curricula in these years. 

3. Analysis of Written Examination Papers 

Written examination in this medical school have traditionally been open-ended (long and 
short essay) and fixed-response (MCQ). In 1990, examination papers of each type, set in 
all courses, were analysed separately for old and new curricula, to determine frequency of 
content areas examined as in a previous study (Rotem et al., 1982). Content areas were 
classified according to organ systems, conventional topics and those of current importance 
to Australian health care. The purpose was to determine content areas which were over- 
or under-emphasized in the curriculum. 

4. Objective Structured Clinical Examination (OSCE) 

This study determined the extent to which third year medical students learned to apply their 
basic science knowledge in the clinical situation - one of the objectives of the curriculum 
change and of the clinical studies programme in Year 3. The OSCE consisted of five pairs 
of stations, each with a test of basic clinical skills and application of basic sciences to that 
clinical skill. The test was conducted on a sample of third year students at the beginning 
of the year and repeated towards the end of the year, before their end-of-year examination. 
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Differences in pre- and post-test scores of those students who sat both tests were 
determined. Perceptions of participating students and examiners about the method of 
examination were surveyed. 

5. Process Observation of Clinical Examinations 

Some concern had previously been expressed about the manner in which clinical 
examinations (long and short cases) were conducted in Year 4 of the old curriculum. The 
research team observed the clinical examiners at the end of the last 4th Year of the old 
curriculum. An observation schedule was used to collect data on student examining 
practices in all four teaching hospitals. The results of the survey were fed back to the 
Integrated Clinical Studies Committee. 

Outcomes and Discussion 

The multidimensional approach to evaluating the changing curriculum enabled curricu- 
lum process and product to be evaluated in quick succession for a given year. Thus process 
evaluation of Year 2 of the new curriculum had first been undertaken when its products, 
Year 3 students, were evaluated, through the OSCE, on their ability to apply what they 
learned in the basic sciences to clinical situations. Awaiting product evaluation to warn that 
something is wrong before undertaking process evaluation to determine what is wrong has 
certain drawbacks. Firstly, the process may have changed by the time deficiencies in the 
product become evident. Secondly, even if the process has not changed, its weaknesses 
would have affected many classes of students before corrective measures are taken. As 
Cronbach (1966) stated, "Evaluation used to improve a course while it is still fluid 
contributes more to improvement, than evaluation to appraise a product placed on the 
market." Deficiencies detected through the OSCE, for example, were pointed out to 
preclinical staff, so that corrective measures could be taken immediately for the succeeding 
group of students. 

Another advantage of multidimensional evaluation was the ability to cross-check findings. 
Thus, for example, gaps in examined content areas, identified through analysis of the 
question paper, could be corroborated through the modified Delphi survey. 

With the limited personnel resources at our disposal, the modified Delphi survey could be 
undertaken only on one year of the new curriculum at a time. Thus the first cohort of six- 
year students have participated in the survey in successive years over the past four years, 
and would continue to do so until they graduate. It has often been stressed that, in evaluating 
an innovation, care must be exercised in drawing conclusions about the new programme 
based on an analysis of the initial class. The "pioneer effect" of the first class is analogous 
to the well known Hawthorne effect in industry. Ideally, until an innovation has been 
stabilised successive classes should be evaluated. The other drawback of our strategy is the 
"fatigue" that could develop in the same group of students surveyed year after year. 
Fortunately, our experience has belied this thus far. The response rate has been high, and 
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students have expressed their willingness to provide feedback on the year they have just 
completed, even though they are aware that any changes would not affect them. The 
advantage of the strategy is the immediacy of the feedback, enabling course coordinators 
to take corrective action if they so wished. The student survey was so timed that it lacked 
threat, though some implicit coercion may have resulted through questionnaire 
administration at the time of enrolment for the subsequent year. 

Providing teaching staff with an opportunity to comment at each stage of the evaluation 
process should have engendered in them a feeling of ownership and commitment. 
Occasionally, however, some staff criticized the questions asked after the results were 
known . even though they did not object to the draft questionnaire before its administration. 
Such criticism, particularly obvious when results were unfavourable, could have been a 
defensive reaction to negative findings. The extreme form of defensive reaction was 
occasionally expressed in statements such as: "These are merely student perceptions'", or 
"How valid are they anyway?" In spite of such comments, however, staff are concerned 
about negative student perceptions and do take steps to address them in succeeding years. 

The curriculum was changed for specific reasons. The evaluation project compared old and 
new curricula to determine whether the change resulted in the desired effects. Evaluating 
corresponding years of each curriculum, while permitting such comparison, also enabled 
deficiencies detected in a given year of the old curriculum to be addressed when the 
corresponding year of the new curriculum was first implemented. As the new curriculum 
progressed, however, a lack of equivalence between the corresponding years of the old and 
new curricula made comparison difficult and even meaningless. 

The strategy of using "marker questions" to compare the abilities of students in the old and 
new curricula was borrowed from a method used by the Royal Australasian College of 
Surgeons to maintain uniformity in minimum pass levels in the Part 1 examination (Cox 
and Royle, 1 980). In those examinations, which are composed of multiple-choice questions, 
at least one-third of the questions are repeated from previous examinations. In our study, 
repetition of questions in successive years was not feasible to that extent. Understandably, 
most examiners were reluctant to repeat too many questions, and a few examiners any at 
all, from the previous year. The small numbers of questions precluded statistical comparisons. 
Lower mean scores in the new curriculum pointed to those areas which required closer 
scrutiny. 

With the lengthening of the new curriculum the sequence of subject areas changed. This 
change, together with longer clinical exposure, made establishing equivalence between 
specific years of the latter parts of the old and new curricula increasingly difficult. The 
"marker questions" study was thus abandoned after the third year of the new curriculum. 

Clinical teaching staff were regularly reminded of this lack of equivalence. In a given year, 
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their expectations of students had, necessarily, to be different in the new curriculum from 
what they were accustomed to previously. Such differences also have important implications 
for external examiners who examine students in curricula which are different from those 
they are accustomed to. External examiners must familiarise themselves with the objectives 
of the course in which they examine. 

The real objectives and priorities of teachers are often evident from the areas in which they 
examine students. This assumption underlay the analysis of written examination papers set 
in the previous five years. From this study over- and under-emphasized content areas were 
identified across the whole curriculum. Rarely do staff consult each other across departments 
in setting examinations; often they are unaware of areas examined by others. As a result 
an appropriate balance in the examination of subject areas is left to chance. This study also 
determined the extent to which emerging health needs were examined. Students are known 
to learn towards examinations. They review previous question papers and seek advice from 
their senior colleagues to obtain clues about the idiosyncrasies of particular examiners. 

In general, this study found that the written examinations set in the school, across the 
curriculum as a whole, were well balanced. Some significant gaps were, however, revealed 
and were a source of concern. It is hoped that, as the new curriculum becomes established, 
these deficiencies would be rectified. 

One objective of the change was to develop students 9 skills in the application of basic 
sciences to clinical settings. The OSCE study was devoted specifically to determine 
whether the new curriculum was achieving t*\is objective. 

The third year marks a watershed between the pre-clinical and clinical phases. A clinical 
course in the third year, labelled Clinical Studies III, has two major objectives: the 
development of basic clinical skills and of the ability to apply the basic medical sciences, 
learned in the first two years, to clinical situations. The OSCE pre-test revealed an inability, 
on the part of the average student, to demonstrate these skills at the beginning of the third 
year. Deficiencies in application were disappointing as, ideally, some vertical integration 
should have taken place in the pre-clinical years. Students learn basic sciences best when 
they see the relevance of the sciences to their future professional life, through application 
and problem-solving. Nevertheless, it is important, at least by the end of the third year 
before the basic medical sciences are forgotten, that the clinically relevant components are 
reinforced through application. The OSCE was standardised, as far as possible, using 
similar patients and simulated patients, the same hospital setting, and at least one examiner 
in a pair who participated on both occasions. Comparison was made on the mean 
performances of only those students who participated in both examinations. 

The OSCE study had a hidden objective. Though this technique had previously been tried 
out, mainly as a teaching device, it had not been established as a formal procedure in the 
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school's repertoire of assessment devices. The OSCE was used as a trial in the third year 
to determine its feasibility and acceptance to staff and student The overwhelming majority 
of staff and students who participated in the examinations expressed satisfaction with the 
method and the desirability of continuing it in future years. The Faculty has now accepted 
it as a method of assessing students in the Clinical Studies III course. 

Both the OSCE study and the process observation of clinical examinations in Year 4 
demonstrated how difficult it is to change practices ingrained in senior clinical examiners. 
Nevertheless, it is our belief that the only path to change is through regular feedback and 
guidance. The improvement in examination technique from the OSCE pre-test to the post* 
test was obvious. It is expected that this improvement would be carried over to the Year 
4 clinical examinations. The observational study of the last Year 4 of the old curriculum 
was preparatory to an improved clinical examination in Year 4 of the new curriculum in 
the following year. As a result of this study a set of guidelines was developed for Year 4 
clinical examiners. The study was not repeated to determine the extent to which improvement 
in technique had really occurred. 

The evaluator's task usually stops with reporting results to the relevant audiences. 
However, especially in the case of internal evaluations such as this, it is imperative for the 
evaluator to assist the decision-makers in the use of the findings to make desirable changes. 
In this study one of the authors was a member of the Faculty Curriculum Committee and 
of two of the Year Committees. In addition he was invited, by virtue of his role as Chairman 
of the Course Evaluation Committee, to all meetings of the Year Committee whose courses 
were being evaluated in a given year. Such attendance was mutually valuable, from the 
point of view of clarifying findings and suggesting alternatives, and for planning future 
evaluation activities through insights into Committee decisions. 

The manner in which committee members reacted varied. Some considered the results item 
by item, discussed possible reasons and suggested ways of improving the following year. 
Others assumed a more defensive posture, attempting to justify weaknesses rather than 
attempting to correct them. Still others were sceptical of student feedback, dismissing it 
as mere perceptions of those who "really did not know what they need". 

Several positive outcomes resulted from the evaluation. Firstly, a Faculty-wide awareness 
of the importance of evaluation was generated. This can largely be attributed to the high 
priority given, and seen to be given, to evaluation by the senior administrators of the 
School. Secondly, there was more interaction on auricular issues among teaching staff of 
different departments. As a result, they became increasingly aware of each other's 
contributions to the curriculum. Thirdly, definite changes took place as a result of the 
evaluation, though more limited than expected. Sometimes these changes were attributed 
to the department 9 s internal evaluation rather than to the CEC s project. The CEC insisted, 
however, that in its final report all evaluation findings were considered together, irrespective 
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of source. On this basis, any positive change brought about was considered worthwhile. 

On one occasion at least an attempt was made to misconstrue an evaluation finding in such 
a way as to achieve a department's hidden agenda. Students' perception of "work 
overload" in a particular subject was construed as "inadequate time" for that subject, and 
attempts were made to increase the curriculum time. This would have gone against one of 
the principles of the curriculum change and had to be resisted, lest the primary objectives 
of the curricular change were perverted. 
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Abstract 

The increased interest, in North America and around the world, in problem-based and 
community-oriented medical curricula has sparked interest in the evaluation of these 
innovative programmes. In January 1989, the Josiah Macy Jr. Foundation sponsored a 
conference to consider designs for evaluation studies and the potential distinctive 
outcomes of the innovative curricula that might be foci of these studies. After defining an 
"innovative curriculum 9 ', the participants identified seven characteristics of "important 
evaluation studies", particularly endorsing studies that compare curricula as whole 
entities. The participants then identified 26 areas where differences between graduates of 
innovative and traditional curricula might be expected, and five equally important areas 
where differences are not expected. Distinctive outcomes of innovative curricula were 

0 Reprinted with the kind permission of the Editor from the paper of this title in Academic 
Medicine (1990), 65, 8-14. 
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anticipated in areas such as interpersonal skills, continuing learning, and professional 
satisfaction. Overall, these recommendations are offered to stimulate creative evaluations 
of the growing number of innovative programmes in medical education. 

In recent years, medical education has been enriched and challenged by a number of large- 
scale curricular innovations. Whereas in the 1 950s the programme at Western Reserve was 
essentially the lone alternative to the conventional model, many North American schools 
are now experimenting with the curriculum in varying ways. Some of these innovative 
programmes - at Brown, McMaster, Michigan State, New Mexico, Rush, and Southern 
Illinois - have existed for many years. More recently, innovative programmes have been 
established at Bowman-Gray, Calgary, Harvard, and Sherbrooke. Looking beyond North 
America, schools at Beer-Sheva in Israel, Newcastle in Australia, Maastricht in The 
Netherlands, and Suez Canal in Egypt, among others, have well-established curricula that 
are clear departures from convention in their respective nations. Worldwide, many of these 
innovative approaches fall under two labels - "community-oriented" and **problem-based" 
- but these rubrics by no means typify all of the models in place. Schools with community- 
oriented or problem-based programmes have established an international network to 
promote attainment of their common goals through joint efforts and information exchange 
(Kantrowitz et al., 1987; Richards et al., 1989). 

The established innovative programmes attempt to address problems of contemporary 
medical education highlighted in recent reports (Muller, 1984; Gastel et al., 1989; Bok, 
1984; Friedman et al., 1983). For example the procedures of problem-based learning 
reduce the emphasis on passive learning of detail during the preclinical phase of education. 
Yet such changes in the educational process, however salutary they may appear, do not 
ensure the desired outcomes of these changes. These outcomes, as described for problem- 
based learning, include greater ability to integrate the basic and clinical sciences and 
superior independent learning skills (Barrows, 1983). Studies are required to determine 
whether the hypothesized linkage between processes and outcomes actually occurs. 

Although there is much interest among North American medical educators - in the 
problem-based approach, particularly - over 90% of North American schools operate 
traditional programmes. Although curriculum change is a deliberate process for many 
reasons, demonstrating positive <5ducational effects at the medical schools where the 
innovative approaches are used may be a necessary condition for widespread dissemination 
of these approaches to other sites. For many constituencies, these positive effects would 
need to be enduring and not counterbalanced by negative effects in other areas. Medical 
faculties universally trained in the scientific method may be particularly inclined to 
demand empirical evidence before undertaking a curriculum change of significant 
magnitude. 

Determining empirically whether these innovative curricula produce a "different kind of 
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doctor" is the role of programme evaluation. There appears to be general agreement that 
such evaluation studies should be done, and indeed, many schools have studied various 
aspects of their own programmes (Schmidt et al., 1987; Woodward et al., 1988; Kaufman 
et al., 1 989). There is less consensus regarding what should be studied and how. The work 
performed thus far has sprung largely from local needs and interests and from the methods 
preferred by the researchers at each school. This is understandable, as many of the studies 
have derived from needs to establish the legitimacy of each innovative programme. But 
this stage seems to be past. With McMaster, Maastricht, New Mexico, and others leading 
the way, the ability of these curricula to produce competent physicians has been generally 
established (Kaufman et al., 1989; Woodward, 1984; Verwijnen et al., in press), although 
further studies of this question may still be desirable. Shorter-term concerns about survival 
of these innovative curricula are being supplanted by longer-term interest in understanding 
their unique effects. 

A Conference on Evaluation 

There is need for a coordinated approach to evaluation of innovative medical curricula. 
This need relates to the shifting of focus from legitimacy to distinctiveness, which allows 
schools to explore issues of more general as well as primarily local interest. It also stems 
from the difficulty of conducting evaluation studies that will be credible in the scientific 
community (House, 1 980). To generate results that faculty and administrators - both within 
and outside the innovative schools - will find persuasive, studies must address important 
questions and employ carefully conceived methods matched to these questions. It follows 
that evaluations will be expensive and time-consuming. Coordination among researchers 
conducting or planning to conduct such studies can generate findings that highlight key 
issues and are comparable across settings. 

To help focus and coordinate programme evaluation efforts, the Josiah Macy Jr. Foundation 
sponsored a conference hosted by the University of North Carolina School of Medicine in 
Chapel Hill. The conference involved the eight individuals who are the authors of this 
paper. While the participants were based exclusively at North American institutions, 
several are well-versed in innovative curricula operating elsewhere through their personal 
activities and through their association with the international Network of Community- 
Oriented Educational Institutions for Health Sciences (Kantrowitz et al., 1987; Richards 
et al., 1989). Planning for the conference began in October 1988; the group met January 
6 and 7, 1989, to generate the recommendations reported here. 

The specific aims of the conference were: 1 ) to consider designs for systematic, comparative 
studies of students trained using innovative curricular models and their counterparts in 
traditional curricula; 2) to identify distinctive outcomes of innovative curricular models 
that might be the focus of these studies; and 3) to determine which studies should ha^e high 
priority for future efforts. 
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In addressing these aims, the participants first considered what it means for a curriculum 
to be innovative and what characteristics of such programmes would give rise to a different 
product. Attention then turned to general characteristics of evaluation studies that would 
justify the time and effort to do them well. With these consensus points as a background, 
a set of more specific recommendations was generated. One is a listing of areas where 
differences between innovative and traditional curricula might (and might not) be 
expected, with a special focus on clinical problem solving. The other recommendations 
identify five high priority areas for study and suggestions for study designs for each of these 
areas. 

What follows is a consensus statement reflecting the best judgment of eight individuals 
who share interests in curriculum innovation and programme evaluation. The 
recommendations are offered in the spirit of suggestion and stimulation rather than 
prescription. 

Defining an Innovative Curriculum 

From the outset, theparticipants recognised the need to setboundariesfor their deliberations: 
to identify the domain of current and future curriculum innovations that would be the 
objects of their recommendations. This is necessary because all medical schools innovate 
to varying degrees, in different subject areas, at different times. Also, innovation is a 
profoundly relative term. What is innovative at one institution, or to a particular set of 
members within that institution, may be recognised as established practice by others. 

Programmes termed "innovative curricula** for the purpose of these recommendations 
have the following characteristics: 1) they operate on the scale of the entire curriculum 
leading to the degree in medicine, or a significant fraction of it. A programme addressing, 
in toto, the two years of preclinical education would qualify by this standard; 2) they are 
guided by some stated, coherent sense of purpose. At some institutions, the purpose is 
conveyed through labels such as "problem-based learning** or "community orientation**. 
Other schools have developed new curricular models more eclectically , adapting for their 
own needs elements of problem-based and community-oriented approaches without 
adopting the terms. Nonetheless, in every school operating an innovative programme there 
is an explicit set of goals guiding the effort; and 3) they involve all medical students or an 
identifiable subset of students at a school If a programme involves all students, it must 
stand as a clear point of departure from standard educational practice in that countiy . If it 
involves a subgroup of students, it must involve a group that remains intact with a distinct 
identity. 

At least ten schools in North America operate programmes with these characteristics. The 
characteristics are stated in generic terms to focus attention on the principles underlying 
the programmes rather than the specific schools that host them, and in recognition that this 
number may change. 
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Educational Context of Innovative Programmes 

Before proceeding to recommend specific designs for evaluation studies, the participants 
explored some basic questions: Why should these innovative programmes be expected to 
produce distinctive effects? What attributes would give rise to such effects? Since decades 
of research have shown that distinctive outcomes of alternative educational approaches 
tend to be weak and ephemeral (Dubin et al., 1970; McKeachie, 1978), there are ample 
grounds of scepticism about the current innovative programmes in medicine. In considering 
these questions, the conferees noted a significant feature common to the programmes 
within the domain of its deliberations. Each programme creates a distinctive educational 
context that is a composite product of the admissions criteria that determine the character 
of the student body; the values of the faculty, who often self-select into the programme; 
the formal and informal educational activities that are part of the curriculum; and the 
character of the communication and personal relationships among students, faculty, and 
programme administrators. The importance of the context, milieu, or total environment of 
a programme has been noted by writers on programme evaluation (Hamilton et al., 1977). 
By creating a complete and alternative educational environment, innovative programmes 
can engender significant effects expressed in the knowledge, skills, behaviour, and 
attitudes of their graduates. 

The distinctive effects of innovative curricula, detectable through empirical studies, will 
result from the total context of the programme. Wherever possible, evaluations should 
focus on these effects of the programme as a whole. Less important are reductionist studies 
seeking to ascribe specific differences in outcomes to specific aspects of the educational 
programme. (For example, are graduates more "humanistic" because of the programme's 
admission policy or because of the community-oriented educational activities?) Many 
proposed studies have been criticized because they cannot isolate the specific aspect of the 
programme that is generating an effect. The present recommendations would temper these 
criticisms. Each element of the programme works in harmony with other elements to create 
the context The context is the direct source of the effects. 

Characteristics of Important Evaluation Studies 

Narrowing the alternatives is a major challenge to any programme evaluation. Educational 
programmes are complex and can be studied from a virtually infinite number of perspectives. 
The conference group identified the following as characteristics important to evaluation 
studies: 

1. Emphasize outcomes. Studies should explore whether the innovative programmes 
generate a distinctive product, as reflected in the behaviour of trainees themselves or the 
health of the patients they care for. Differences in educational process are of interest to the 
extent that they contribute directly to desired educational effects. 

2. Explore effects that are reasonably enduring. An effect that exists ephemerally, only 
at the end of the innovative phase of training, is of less interest than an effect that persists 
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months or years later. 

3 . Can be concluded within a decade. Many studies of interest are necessarily longitudinal, 
but ten years is as long as the community can be expected to wait for important findings. 

4. Are of interest to those who will make decisions about the continuation or adoption )f 
tfiese educational approaches. Researchers with training in the behavioural or social 
sciences might value measures that connect to the scholarly literature on human learning; 
however, those who control resources and access to the profession may have primary 
interest in measures with close ties to institutional administration and activities of 
professional practice. 

5. Have potential to generate reasonably large effects. Longitudinal studies can involve 
many hundreds of subjects. With such sample sizes, effects of little practical interest can 
still be significant statistically. The variables studied should be those where reasonably 
large, rather than merely statistically significant, results are expected. 

6. Have not been undertaken already. With the large number of interesting studies that can 
be done, the field cannot yet afford studies that are purely replications of previous work, 
in the absence of clear justification. 

7. Cannot be performed routinely. A great deal of data that can be used in evaluation are 
generated by the routine operation of a programme. These routine data can be used in 
curriculum evaluation but they cannot alone answer tks important questions. The most 
informative studies will employ data collected as part oi specially designed, prospective 
research efforts. 

Studies in accord with these guidelines will enhance understanding of innovative 
programmes and inform decisions on potential adoption of these programmes by other 
schools. The results may also help schools with innovative curricula make theirprogrammes 
more effective. 

Where Differences Are and Are not Expected 

In curriculum evaluation, demonstrations of "no difference" are often as important as 
demonstrations of differences. If, for example, the general belief among educators is that 
the innovative programmes are more costly per student, a finding that the costs are 
equivalent will be of great interest even though, in experimental terms, this is a null result. 
For this reason, the conference group directed attention to both kinds of outcomes: where 
differences are, and are not, expected. 

As shown in Table I, the deliberations generated a list of 26 specific areas in which 
educationally significant differences might be expected when comparing students or 
alumni from innovative and traditional curricula. The anticipated differences can be 
grouped into six clusters, also given in the table. This list is suggestive rather than 
exhaustive, intended to direct attention to a useful set of targets for evaluation studies. 
Individual items in the list will be more applicable to some curricula than others. The 
differences are listed without an assertion of directionality, although most studies will 
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Table L Hypothesized areas of difference between traditional and problem-based and/or 
community-oriented curricula 



Psychosocial and Interpersonal Skills 

1. Sensitivity to individual patient needs 

2. Interest in primary prevention; awareness of and ability to use community resources 

3. Appreciation of the effects of a disease on the patient's family 

4. Sensitivity to the context of a patient's life; for example, occupational risk factors, his/her social 
group and role in the community 

5. Ability to work efficiently and effectively in a diverse team 

6. Degree of competitiveness within the medical student body. 

Continuing- Learning 

7. Keeping up to date in both traditional and innovative ways; modifying practice activities in 
accord with changing biomedical knowledge 

8. Mastering and maintaining skills related to information access and information retrieval 

9. Habitually filling perceived gaps in medical knowledge. 

Professional Satisfaction 

10. Levels of student and faculty satisfaction with their circumstances in medical school 

1 1. Rates of burnout, dropout, impairment, and mental health problems among practitioners 

12. Practitioners* satisfaction with their careers, particularly the day-to-day work of clinical 
practice. 

Practice Beliaviour 

13. Amount and judiciousness of prescribing 

14. Initiative in maintaining a programme of patient care; for example, by active follow-up to ensure 
compliance 

15. Referral patterns (which problems are referred and which are treated by graduates in primary 
care) 

16. Proportion of graduates who practice primary care 

17. Extent to which graduates conduct research in the context ot their patient care activities 

18. Extent to which graduates participate in community activities 

19. Extent to which graduates are "risk takers" or advocates for social issues affecting health. 

Educational Achievement and Cognitive Development 

20. Scores on board examinations and other tests of knowledge* 

21. Knowledge of the basic sciences when that knowledge is displayed in the context of clinical 
problems 

22. Rate of professionalisation (socialisation into the profession) 

23. Rate of ethical and cognitive development. 

Institutional Issues 

24. The popularity of the school and programme as interest in medicine generally declines 

25. Extent of alumni contact with and loyalty to their school 

26. Differences in the faculty value system, particularly with regard to education as a component 
of the institutional mission. 

*A difference favouring traditional curricula is posited here. 
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likely be conducted with the expectation of finding differences favouring the innovative 
programmes. However, with regard to performance on traditional standardised tests of 
basic science knowledge (such as the NBME Part I), the group 9 s general expectation is that 
students in traditional curricula will outperform those in innovative programmes. Some 
differences that exist at the completion of the medical curriculum may be negated by 
postgraduate training; where this is expected, studies should be conducted at several times. 

The group also identified five areas where differences are not expected, as summarised in 
Table II. Taken together, these items imply that both innovative and traditional auricular 
models will produce graduates prepared for further clinical training and ultimate practice, 
and that both will be equally costly in the steady state. However, cost may be a differing 
function of class size for the two types of programme. Costs per student for traditional 
curricula will decrease as class size increases, while comparable costs of innovative 
curricula may increase with increasing class size. While the programmes may be equally 
costly when viewed over all schools, an individual school may find one type of programme 
more costly, depending on the school's class size. 

For studies of innovative curricula, the "no difference" hypotheses listed in Table II are as 
important as the areas of potential difference posited in Table I. Studies directed at clinical 
problem solving of common medical problems are a special issue discussed in detail 
below. 



Table II. Hypothesized areas where no differences are expected between traditional 
curricula and problem-based and/or community-oriented curricula 



1. Passing rates on licensure and certification examinations 

2. Likelihood, in practice, of making a major clinical mistake 

3. Clinical problem solving around common, uncomplicated presentations of disease 

4. Educational costs per student (in the steady state, after start-up, costs of innovative 
programmes attenuate) 

5. Cost-efficiency of graduates as practitioners. 



Clinical Problem Solving 

The conference group viewed clinical problem-solving proficiency as the ability to "work 
up" efficiently a patient with a common medical problem, reaching an accurate diagnosis 
and formulating an appropriate plan for management. It is tempting to hypothesize that the 
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innovative curricula would increase students' problem-solving proficiency. In community- 
oriented curricula, significant clinical experience begins early in training with a focus on 
common problems and primary care. In problem-based curricula, the first and second years 
are based on a sequence of clinical problems. Although these problems tend to be paper 
cases rather than live patients, the problem-based approach introduces clinical reasoning 
-the use of objective and subjective findings about a patient to infer the presence of specific 
diseases - at a very early stage. In contrast, in traditional North American curricula, 
significant clinical training does not begin until the third year and the most common setting 
is the tertiary care hospital. 

Use of the traditional model is supported by what may be termed a "trickle down" theory 
of learning in clinical medicine. According to this theory, complicated medical problems 
seen in tertiary care centres are superpositions of simpler problems. It is presumed that a 
student who can manage a complex problem in a tertiary setting can also manage the 
individual components of this complex problem as they arise in primary care settings. 
Innovative models for clinical curricula run counter to this assertion, arguing that it is 
difficult for students to separate the components of a complex problem in a very sick patient 
and thus "obtain valid impressions of the over-all health problems of the community" 
(White et al., 196 1 ). Students have less appreciation of diabetes mellitus as it usually occurs 
in the population if they interact only with patients who have diabetes with extensive 
complications. This view leads to an emphasis on training in settings where students are 
exposed to problems to the extent that, and in the form that, these problems occur in the 
general population. 

On balance, however, there should be no enduring differences in basic clinical problem 
solving between students in innovative and traditional programmes. Students develop 
proficiency in areas where they get more practice and exposure. Basic problem-solving 
skills will likely be equivalent across programme types, not because these skills "trickle 
down" from tertiary care experience in traditional schools but rather because all curricular 
models try to provide an adequate level of direct exposure to uncomplicated presentations 
of common medical problems, and most succeed. While students who graduate from these 
various programmes are not likely to differ in problem-solving proficiency across large 
numbers of common medical problems, students within schools may differ idiosy ncratically 
as a consequence of the specific clinical experiences they are assigned or elect. This is, 
furthermore, an area where any differences existing at graduation will attenuate during 
residency. 

Recommendations for Outcomes of Highest Priority 
Tables I and II list a large number of outcomes of potential importance in studies comparing 
innovative and traditional curricula. In a world of limited time and resources, priorities 
distinguish what actually occurs from what is merely planned; therefore, the group 
concluded its deliberations by considering five outcome areas that appeared deserving of 
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Table m. Specific recommendations for evaluating five important o»je C omes of curricula 



Outcomes 



When studies should Proposed methods 
be done 



What is needed most 



Psychosocial and inter- 
personal skills cluster 



When students and gra- 
duates are working with 
patients 



Continuing learning 
cluster 



Satisfaction cluster 



Clinical problem solving 
(No difference expected) 



After five or more years 
into practice 



Cost of education 

(No difference expected) 



At regular intervals 
during school and after 



• Simulate "difficult" 
patients or patients 
with psychosocial 
problems 

• Conduct surveys irk- 
ing patients to assess 
the psychosocial skills 
of practitioners 

Find out: 

• Do they know the 
latest? 

• How do they keep up? 

• Rates of practice 
change and why they 
change 

• Do they influence the 
practice behaviour of 
colleagues? 

Study scores on: 

• Recertiflcation exams 

• Technical skills exams 

Interview and mail sur- 
veys 



Since many measurement 
techniques exist, design 
and logistics of studies is 
the major challenge - 
particularly obtaining 
large sample sizes 



Studies using a variety of 
methods and having 
enough subjects at enough 
sites to be informative 



More studies of faculty 
and practitioner satis- 
faction since most schools 
are already addressing 
student satisfaction 



Early in clinical training, 
after clerkships, at gra- 
duation, perhaps after 
internship; once trainees 
specialise, a generic 
exam is no longer practi- 
cal 



Various types of clinical A consensus on what are 
simulations and vignette- "common problems"; it is 



based multiple-choice 
questions 



not clear schools will 
agree when specific case 
problems are designed 



During start-up phase of Time- and -motion studies 

a programme, then con- of faculty and staff acti- 

tinuing at least five years vity, so their time devot- 

into a programme's ope- ed to education can be 

ration computed 



> Studies comparing direct 
costs of innovative and 
traditional education 

• Studies computing start- 
up costs vs. steady-state 
costs for new program- 
mes 

• Studies examining cost 
per student as a function 
of programme size 
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the most, and the most immediate, attention (see Table HI). 

Two of the five high-priority areas are those where no differences are expected: clinical 
problem-solving and cost of education per student Clinical problem solving, as discussed 
in the previous section, is so fundamental to the mission of medical education that it 
requires continuing exploration. The importance of comparing programme costs is self- 
evident as financial pressures on academic medical centres mount and programmes must 
justify themselves on fiscal as well as educational grounds. Schools considering the 
adoption of an innovative curriculum face substantial planning and start-up costs. These 
start-up costs will be particularly difficult to justify if the steady-state cost of operating the 
new programme is substantially higher than what it replaced. 

The other three high-priority outcomes address areas where differences between innovative 
and traditional curricula are anticipated. These include a cluster of outcomes relating to 
psychosocial factors in patient care and interpersonal skills, a cluster related to continuing 
learning, and a third cluster bearing on satisfaction of students and faculty with their 
education and practice. If the innovative curricula are going to make a difference, the 
participants believe these to be the important domains where the differences will be found. 
It will be important to track these differences over time to see whether they remain after 
residency and into practice. The importances of psychosocial issues (Engel, 1977) and 
continuing learning (Manning et al., 1987) in contemporary medical practice have been 
widely discussed. The importance of satisfaction as a generic dimension of study stems 
from the literature on physician impairment (McCue, 1982; Logan, 1989) as well as more 
recent studies of stress and depression in medical students (Witaliano et al., 1989; Spiegel 
et al., 1986). 

Table m elaborates on study designs for each of the five outcomes, when in the continuum 
of medical education measurements should be taken, methods of measurement or assessment 
that might be employed, and what specific efforts are most needed to complement what is 
already being done. In all study designs, sample size and statistic power emerge as 
important considerations. In areas where differences between programmes are expected, 
sample sizes must be sufficient to detect the differences that are educationally meaningful. 
In areas where no differences are expected and none are found, an adequate sample size 
will lend credence to the results. 

Conclusions 

Four general themes emerge from these deliberations. First, if the innovative programmes 
do engender distinctive effects observable in their students or graduates, this may be due 
to the pervasive context stemming from the values of faculty, students, and administrators 
and reflected in how the educational experience is organised. It is difficult, and it may prove 
impossible, to attribute effects of alternative curricula to specific component parts. 
Evaluation should emphasize comparison of curricula as whole entities. 
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Second, any study worth doing is worth doing well enough to ensure that the results are 
not clouded by conceptual or methodological shortcomings. Each study will, therefore, be 
time-consuming and costly. As a consequence, studies should address important issues and 
should be coordinated across sites to use redundancy to advantage. In general, studies that 
are pure replications should have lower priority than studies that address novel questions. 

Third, the specific curricular outcomes worthy of study include areas where differences are 
not expected, as well as areas where differences are expected. While there is a tendency 
to give second-class status to studies with "no-difference" findings, studies of medical 
problem solving and educational costs are very important in the comparison of traditional 
and innovative programmes of medical education. 

Fourth, the expected distinctive outcomes of innovative curricula fall into categories such 
as interpersonal skills, continuing learning, and professional satisfaction. These are 
extremely important aspects of professional practice and well justified as foci for 
evaluation. 

It is the authors' hope that systematic study of innovative medical curricula, in North 
America and worldwide, will grow as the number and longevity of innovative programmes 
increases. The results of thoughtful, well-designed, well-executed studies, whether or not 
they favour the innovative programmes, will benefit everyone who works to improve 
medical education. 
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Some Reflections on Evaluation of Outcomes of Innovative Medical 
Education Programmes during the Practice Period*/** 

Christel A. Woodward 
MacMaster University 
Hamilton, Ontario, Canada 

Abstract 

Innovative medical schools have reached a stage of maturity where they can begin to 
enquire into the outcomes of their curricula in relation to their graduates and any more 
wide ranging influence on other educational institutions. What questions should be 
addressed that concentrate on major implications and can be expected to endure over time 
and are thus of interest to decision-makers? What questions can be addressed that take into 
account constraints, as well as opportunities? Reality empfiasizes that undergraduate 
education alone does not determine the performance and behaviour of graduates; the 
influences of postgraduate education and those of the practice environment are super- 
imposed. With whom should the graduate be compared? Numerous variables need to be 
considered, and bias, such as that of student selection, needs to be avoided. The use of 
unique local opportunities is illustrated with examples from Canada and McMaster 
University. In the author's experience outcome evaluation is likely to raise as many 
questions as it answers, and it may lead to new questions that are more focused and refined 

Decisions made in the late 1960s and early 1970s to expand the medical school capacity 
in many countries provided a unique opportunity for innovation in medical education 
(Richards et al., 1989; Kantrowitz et al., 1987). Innovative programmes were begun in 
such diverse settings as Beersheva in Israel, McMaster in Canada, and Newcastle in 
Australia. These innovations were often centred on incorporating newer educational 
approaches into the curriculum and attempting to make medical education more responsive 
to perceived societal needs. As a result, the admissions requirements were often expanded 
for medicine to allow greater diversity in the student body, problem-oriented, small group 
learning was used, or more of the education was carried out in community settings. Yet, 
because these innovations were embedded in different cultural contexts, innovation also 
developed in unique ways. Differences were shaped, in part, by the existing traditions in 
medical education and health care delivery in those countries and by the most pressing 

* When a doctor has completed his formal education and practises without supersivion. 

* * Based on paper presented at the Workshop: Evaluating the Outcome of the Undergraduate 
Medical Course, September 199 1 , Faculty of Medicine, University of Newcastle, Australia." 

Requests for reprints should be sent to Christel A. Woodward, McMaster University, Program for Education 
Development, 1200 Main Street West, Hamilton, Ontario L8N 3Z5, Canada. 
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issues in how physicians are selected, educated and eventually practise in their countries. 
Thus, each programme is a product of both international forces and forces in its own 
country. 

In the past decade, these schools have reached the point in their history where sufficient 
numbers of graduates have entered practice to ask questions about the impact of these 
physicians on organised medicine and medical care delivery in their countries. One general 
question of interest is what similarities and differences exist in the outcomes of innovative 
medical education programmes. However, while this general question appears deceptively 
simple, in practice such outcome evaluation is exceedingly complex and difficult. Specific 
questions must be framed which obtain answers that are of interest to decision-makers. The 
methods used in the evaluation must be credible. Further, engaging in outcome evaluation 
may not be worthwhile in some settings if the costs involved are likely to exceed the value 
of the information obtained. 

What Questions should be Addressed? 

Several generic criteria can be used to help to focus an evaluation exercise. Clearly, it is 
important to focus on questions that will advance our understanding of the impact of these 
innovations in medical education. The answers we receive should inform the decision- 
making process. If no actions will be taken (even the decision to maintain the status quo) 
by the answers we receive, the question is not worth asking. Careful thought must also be 
given to whether the answers can be obtained in a timely fashion. If the answers are delayed 
to such a point that they are no longer considered relevant (because of changes introduced 
during the intervening time), they cannot inform decision-making. Similarly, finite 
resources are usually available for evaluative research. Choosing to answer some question 
involves opportunity costs. Other questions, or even other ways in which the money could 
have been spent aside from outcomes research, are foregone. The perceived potential cost 
of not obtaining the information may also be considered. Often, the exercise becomes one 
of deciding which of several interesting questions is most worthy of pursuing, given the 
existing resource constraints. 

The outcomes of interest must be measurable in practice. Practical ways to operationalise 
the constructs of interest must exist that are precise and accurate. Global constructs, such 
as "good physicians" must be broken down into specific, measurable attributes and 
behaviours, if measurement is to be attempted. Further attempts at operational definition 
may reveal that little consensus exists for such definition, because such global constructs 
as producing "good physicians** or physicians "responsive to societal needs" were often 
used to appeal to a wide range of potential audiences at the initial stages of securing support 
for the innovation. 

Questions concerned with outcomes should focus on effects which are hypothesized to be 
fairly large and are likely to have endured overtime. Intervening events (e.g. postgraduate 
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education), constraints imposed by existing structures for medical care delivery, and the 
societal context in which care is delivered should not be able to wipe out the intended effect. 
Several studies have shown that effects of an educational proramme, observed shortly after 
it took place early in medical education, may not last even through the final years of the 
curriculum (Engler et al., 198 1 ). Further, if the length of postgraduate education is similar 
or even exceeds the length of medical school education (as often occurs in North America), 
the type, length and location of postgraduate education may constitute important confounding 
variables in outcome evaluation. If this is suspected, these factors should also be measured 
as part of the study design. 

The existing health care delivery system, particularly the financial reward system for 
physicians, and the context in which medicine is practised may exert a profound influence 
on how physicians behave. Thus, the likehood of observing outcomes of interest may also 
be muted by the values and rewards encountered in practice. For example, in North 
America, it may be exceedingly difficult for physicians to resist ordering unnecessary or 
marginally necessary laboratory tests, when the practise climate emphasizes defensive 
medicine and the judicial system awards large judgements based on lack of exhausti veness 
of testing rather than lack of application of reasonable clinical judgement (Woodward & 
Rosser,1990). In some developing countries, the financial incentives given to specialty 
practise in urban areas may impair the ability of community-based programmes to produce 
primary care physicians willing to serve in less well-equipped settings and less well-paid 
positions in geographically isolated rural areas. The demonstration of such problems, and 
particularly, the effects of interventions to overcome them, may be as important as 
demonstrating that some medical education interventions have an enduring effect. However, 
such studies sh ft our focus away from the enduring aspects of medical education to address 
reasons why effects of medical education do not endure. 

What Kinds of Outcomes of Medical Education are Likely to Endure and 
be Observable during the Practice Period? 

At a recent conference, sponsored by the Josiah Macy Jr. Foundation, we tried to identify 
possible important differences in the outcomes of problem-based and/or community- 
oriented innovations in medical education which may endure (Friedman et al., 1990*). 
Unfortunately, participants were medical education researchers from North America only. 
Our limited experience with innovative programmes in other parts of the world may have 
produced some bias in our assessment. However, with this caveat in mind, the group made 
several specific recommendations about evaluations of outcomes seen as important and 
likely to endure. Three outcomes, seen as measurable during the practice period, were 
differences in 1) psychosocial and interpersonal skills; 2) satisfaction (with medical 
education and carter); and 3) continued learning. The latter outcome was not seen as 
measurable until the physician had been in practise five or more years, while the former 

* This paper has been reproduced in this Volume of the Annals. 
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outcomes were seen as measurable both at earlier points in time and during the practice 
period. 

The group came to recognise that clinical problem-solving was not likely to be an area 
where differences between innovative and traditional medical curricula would emerge 
during the practice period. This was debated within the group at some length. Eventually, 
we decided that the effects of subsequent education (during clerkship and the post-graduate 
period) were likely to attenuate any early advantage achieved by problem-based curricula. 
Further, the construct was seen as not likely to be measurable in practice, given that the 
range and complexity of problems that are seen during that period is often dependent on 
the field of medicine which is chosen by the individual doctor. Performance in problem- 
solving may be content specific (Norman, 1986). By the time physicians entered practise, 
generic problems are likely to be too simple (suffer from ceiling effects) and inappropriate. 
Further decisions about the types of problems used might influence the results, because 
only a very limited number of problems would be likely to be feasible. 

No attention was paid to the choice of specialty, to location of practise or to other aspects 
of practise behaviour, although these may also be seen as legitimate, enduring outcomes 
of some innovative schools. For example, the University of New Mexico's alternative 
track of medical education has a specific objective to increase the number of physicians 
choosing primary care careers (Kaufman, 1989). Community-based schools in particular 
may seek to influence the mix and locations of their graduates (Richards et al., 1989; 
Kantrowitz et al., 1987). However, powerful discentives may operate in the way the health 
care delivery system is organised. Yet, differences in decisions about field and type of 
practice may reflect important differences in the outcomes of medical education. 

With whom should the Graduates be Compared? 

The choice of a comparison group or groups may seem to be a simple decision. Glibly put, 
we are looking for "contemporaries" or "peers** who also went to medical school but did 
not experience innovation. For medical schools that developed innovative, parallel 
"tracks* or had a traditional programme before the innovation began, internal comparison 
groups may possibly exist. Unless selection into the parallel curriculum was random, use 
of contemporaries from the traditional track of the same school may be subject to selection 
biases. Contamination may also have occurred (e.g. the comparison group was exposed to 
many elements of the new curriculum). Use of previous graduates also has some problems. 
If differences are seen, they may be unrelated to the curriculum and simply reflect time 
trends. If no differences were seen, it is possible that contamination occurred. For these 
reasons, neither of these internal comparisons are likely to contribute to designs that 
provide strong evidence about the impact of innovations, unless students were randomly 
allocated to participate in an alternative curriculum and the traditional curriculum. 

Graduates of other medical schools, while somewhat more difficult to identify and recruit, 
are likely to be a better choice for comparison purposes. Again, "peer** may not be 
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completely obvious. Is the external comparison simply someone who graduated at the 
same time? Should graduates of one or more medical schools be included as the 
comparison group? This type of comparison may be appropriate for an examination 
whether or not differences in basic choices (field, length of training and location) have 
occurred. Yet, caution must be used even here. In North America, such basic choices are 
known to be influenced by sex of physician and the age of the physician at graduation 
(David et al., 1 990). Thus, when comparing McMaster graduates with their contemporaries 
on these dimensions, we chose two comparison groups, one matched only for graduation 
year and one where the McMaster graduates were matched by age and sex as well as year 
of graduation (Ferrier et al., 1987). 

Practice behaviours of interest (e.g. attention given to patient education) may be influenced 
by basic career choices. When measuring behaviour in practice, << peers" are more likely to 
be physicians who practise in the same field of medicine. Yet, length of time in practice, 
sex of physician, location and type of practice are also likely to influence practice 
behaviour (Eisenberg, 1985). Thus, in structuring comparison groups, these factors should 
be considered (potential matching variables). If the sampling frame to be used contains no 
information about these factors, efforts must be made to obtain information to evaluate 
these possible confounding variables. 

Using Unique Local Opportunities 

Both decisions on what to evaluate in outcomes research and how to evaluate it are likely 
to be heavily influenced by the opportunities that are available in the setting in which 
outcome evaluation occurs. Certainly, this has been the case in the author's own studies 
of the outcomes of medical education at McMaster (Woodward, 1984, Woodward, 1989). 
The availability of good sampling frames and routinely collected information was a major 
factor in how outcome studies were designed. The ability to use such information as age, 
sex and year of graduation, in order to find matched comparisons, reduced the variability 
in the data and made measurement of effects more precise. 

A sampling frame from the Canadian Medical Association, which had carried out a 
national manpower survey of physicians licensed in Canada in 1983 (Woodward et al., 
1985), was used to select two comparison groups for McMaster graduates (1972-1979) 
from English-language Canadian medical schools. The first comparison group was 
matched by year of graduation alone, while the other was matched by age and sex as well 
as year of graduation. The latter comparison group was chosen because McMaster had 
graduated more women, and its student body was somewhat older than at other Canadian 
schools. Anonymous data from the survey were used to answer such basic questions as 
whether McMaster graduates differ in their choice of fields of medicine and how they 
spend their time (Ferrier et al., 1987). We found that McMaster graduates were less likely 
to enter surgery and its subspecialties, and more likely to enter internal medicine and its 
subspecialties than other Canadian English-language graduates but equally likely to 
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specialise or chose a primary care (general and family practice) career. Among primary 
care physicians, McMaster graduates were more likely to obtain certification from the 
College of Family Physicians of Canada than their contemporaries. McMaster graduates 
also differed in that they were more likely to report involvement in research/education and 
administration as part of their practice, and to hold academic posts than their peers. These 
differences were observed whether one or both comparison groups were considered. 

In another study (Woodward et al., 1990), the sampling frame was the Ontario Ministry 
of Health 9 s file of primary care physicians who billed the Ministry for their servic; Ang 
the 1986 calendar year. The sampling frame for this study included about one-third of the 
output of both McMaster and other Ontario medical schools during 1972-1983. Here 
anonymous data on all McMaster fee-for-service primary care physicians were obtained 
along with a comparison group of graduates of other Ontario medical schools who were 
matched to the McMaster graduates by sex, location and year of graduation. Data on 
certification status had been obtained from the College of Family Physicians and were 
added to the file by the Ministry of Health. Separate information on hours worked during 
1986 was obtained from the Canadian Medical Association which had completed another 
manpower survey of Canadian licensed physicians in 1986 (Adams, 1989). These data 
could not be directly added to the file. 

McMaster graduates in general and family medicine, while working similar hours to those 
of the comparison physicians, saw fewer patients, provided fewer services and earned less 
under the fee for service system and had a somewhat different pattern of practise 
(Woodward et al., 1990). They wfcre less inclined to work in an emergency room and do 
minor assessments (brief, involving only one organ system or problem) while they were 
more likely to offer prenatal care and psychotherapy to their patients. 

Serendipity also can occur. During the selection of a matched sample (by year of 
graduation, sex and location) of McMaster and other Ontario general and family practise 
physicians from the Ontario Ministry of Health's database, we noted that McMaster 
graduates, who are general and family physicians in Ontario, are more likely to choose 
alternatives to fee-for-service funding (capitated and salaried positions) than their 
contemporaries who graduated from other Ontario medical schools (Woodward et al., 
1990). This finding was interesting, because the Ontario government is actively promoting 
alternatives to fee-for-service medicine. It feels that fee-for-service practice rewards the 
number of specific services billed for a patient and the number of patients seen more than 
the extent to which the physician carefully attends to the health care needs of the individual 
patient. 

Using secondary sources of data ensures that the sample will not be biased through limited 
participation by one or both groups. Yet, by not obtaining information directly from 
physicians or their patients, only those aspects of practise about which data are collected 
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can be studied. We can somewhat augment such files by acquiring data from several 
secondary sources (e.g. regarding certification and hours worked). But, whether the 
differing patterns of practice we observed are related to differences in quality of care 
cannot be answered with such data. 

A proposal to examine quality of care was developed at McMaster over ten years ago but 
not funded. The cost was deemed too high for a study to show, as one reviewer put it, "that 
McMaster graduates are safe; we already know that*'. Patient satisfaction, extent of 
agreement between doctor and patient regarding what had been communicated about 
selected problems, practice audit of selected indicator medical conditions, use of selected 
drugs and drug combinations, and the use of standardised patients to assess interpersonal, 
clinical and management skills were some of the measures proposed at that time. 

In the intervening years, we have developed more precise information about differences 
and lack thereof between McMaster graduates and their contemporaries that would allow 
refinement in study design. For example/ we have shown that McMaster graduates in 
general and family medicine are more likely to seek certification by the College of Family 
Physicians (Ferrier et al., 1987; Woodward et al., 1990) and are more likely to include 
teaching activities during their practise. Certification in family medicine among Ontario 
physicians has been shown to be linked with higher quality of care in a recent study by 
Borgiel and his colleagues (Borgiel et al., 1989). This pioneering study developed a 
sophisticated approach to quality of care assessment in primary care. Their strategy for 
recruiting randomly selected physicians to participate in the study involved personal 
contacts by opinion leaders in the physician' s home community and led to high agreement 
to participate among the physician . to be sampled. A quality of care study involving 
McMaster graduates would need to ask whether the differences, if present, are observed 
generally or can be explained by the increased likelihood of certification oi involvement 
in education among our graduates. 

Good descriptive information about how graduates differ from, and are similar to these of 
other schools, is important to study design. For example, we have chosen to focus on 
primary care physicians because they are the largest group of graduates and because no 
difference was observed in the proportion of McMaster and other contemporary graduates 
in the proportion choosing primary care (Ferrier et al., 1987). If far fewer or many more 
McMaster graduates had chosen to enter primary care than their peers, we would be 
concerned about differential selection confounding our groups. Prirrary care physicians 
are more like.'y to remain in Ontario than thosj entering specialties. Mobility of graduates 
also makes i f i.iore difficult to study them. Knowledge about the patterns of dispersion of 
graduates can be important to study design. 

Availability of measurement tools may also create research opportunities. One of our 
colleagues, Brian Hay nes, and a medical student, John Shin, recently undertook a study of 
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continuing learning by McMaster graduates in primary ca.~ in Ontario (compared to 
University of Toronto graduates). The management of hypertension (Shin et al., 1991), a 
field of practise where significant changes in recommended management have occurred 
over the past fifteen years, was the focus of the measurement tool used. They found that 
McMaster graduates correctly answered more questions about the current preferred 
management of hypertension than University of Toronto graduates. A high response rate 
was achieved in both groups (80%). Physicians were invited to participate by a letter from 
the deans of both medical schools and intensive follow-up efforts were made to enhance 
participation. The measure used had been designed for use in a randomised controlled trial 
which assessed knowledge about hypertension in community physicians exposed and not 
exposed to an educational intervention during practice (Evans et al., 1984). It was checked 
and updated using the recommendations of the Canadian Hypertension Society. Thus, the 
first study of the impact of McMaster on the continuing learning by its graduates (Shin et 
al., 1991) used an opportunity available locally - a measure that had already been 
developed for another purpose and one that needed only minor updating. Because a faculty 
member was able to interest a medical student in collaborating on the project, it became 
a research experience for the student, and study costs were reduced. 

This pioneering study suggests that McMaster graduates may be more effective in 
continuing learning during practice than graduates from a traditional medical curriculum. 
However, it is possible that the same difference may not be observed for other topic areas. 
Our Faculty has long had a major research interest in the management of hypertension, and 
this may have contributed to the effect observed for this topic. Only a series of studies 
demonstrating more up-to-date knowledge, across areas of medicine, would be convincing. 
The extent to which physicians are involved in teaching students may influence how up to 
date their information is. We already know that McMaster graduates are more likely to be 
involved in medical education and, among those who are involved, spend more hours 
teaching per week (FerrieretaL, 1987). Thus, future studies are likely to ask more refined 
questions. It becomes importantjtojcnow whether any effect observed is an indirect effect 
(a function of the higher involvgftcnt in education among McMaster graduates) or is a 
direct effect (observed among McMaster graduates generally, after controlling for their 
increased likelihood to teach). 

Unique opportunities are likely to exist in other countries to answer some questions 
regarding innovations in medical education. For example, we cannot unravel selection 
from curriculum effects when assessing the impact of the educational intervention at 
McMaster (W oodward, 1 984). Both the selection process and curriculum differ from other 
medical schools in Ontario. Further, the selection process is designed to identify learners 
who will do well in our curriculum. However, selection of medical students in The 
Netherlands occurs by lottery among those who are eligible (Verwijnen et al., 1989). This 
allows the Uni veristy of Limburg at Maastricht to examine the effects of curriculum alone. 
Newcastle in Australia also has a unique opportunity to examine whether selection, 
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curriculum or both contribute to outcomes. Half of their medical class is selected solely on 
the basis of their past academic record, while the remainder are chosen by applying 
carefully defined criteria related to academic, intellectual and personal attributes (Maddison, 
1989). 

Conclusion 

Assessment of outcomes of innovative medical schools among practising physicians 
requires that we examine skills, behaviours and attitudes that can be expected to be long 
lasting and will be of interest to decision-makers. Such outcome assessments should focus 
on effects that can be measured in a timely fashion and are large enough to be observed, 
given the precision and accuracy of existing measurement tool s. The possible confounding 
effects of intervening and concurrent experiences must be considered. 

Types of outcome evaluation that can be carried out depend on the ingenuity and 
imagination of the local research groups in exploiting the unique opportunities available 
in their setting and the way they handle constraints. Constraints on outcome evaluation 
may include the lack of availability of sufficient financial and human resources, lack of 
national or regional information about physicians and their practices, and absence of 
measurement tools for the outcomes of interest. The willingness of graduates, and those 
with whom they are to be compared, to participate may also act as a constraint. Mobility 
of graduates may also serve as a constraint, as those who reside outside the area or country 
are more difficult to include in studies. Increased sophistication in the assessment of 
quality of care makes this type of measurement more feasible, if this is seen to be an 
important outcome of the innovation. Study design may be refined by the availability of 
sampling frames which allow selection of graduates for comparison who are matched for 
potential confounders. 

In our experience, outcome assessment is likely to raise as many questions as it answers 
and may lead to new questions that arc more focused and refined. 
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Abstract 

Findings from a continuing programme of evaluation research at the University of 
Limburg are presented. The research programme aims to clarify to what extent problem- 
based learning is an effective and efficient approach to medical education and if so: Why. 
The data gathered over the years suggest that there is no particular reason to believe that 
problem-based curricula provide inferior training to medical students as compared with 
conventional curricula. Medical students in a problem-based curriculum do not spend less 
time on their study, and their achievement is comparable to that of students of traditional 
schools. Problem-based curricula do appear to provide a friendlier and more inviting 
educational climate, which, in turn, creates a comparatively lower dropout rate. There is 
some evidence now that students in a problem-based curriculum integrate knowledgefrom 
differentdomainsbenerarulacquiresuperiorreasoning 
curricula. 

This paper reviews findings from a continuing programme of evaluation research carried 
out at the University of Limburg. The research programme attempts to clarify to what 
extent problem-based learning is an effective and efficient approach to medical education 
and if so: Why. We present data on medical students' attitudes toward instruction, student 
achievement in terms of test scores measuring medical knowledge, and problem-solving 
capabilities acquired by students. Comparative data on duration of study, dropout rates, 
and study load are also presented. 

In this discussion, we capitalise on a unique aspect of the Dutch medical student selection 
system: the lottery . This process allocates students by chance not merit or other considerations 
to various medical schools after baseline requirements have been met. The student bodies 
of the medical schools in The Netherlands are thereby more similar than is typically the 
case in the rest of Europe or in the United States. This is a boon for comparative evaluation 
research of the kind reported here. However, as selection of the students in the studies 
pitsent is not always entirely random, the comparisons - although scientifically justified 
- need to be interpreted with some caution. 



Requests for reprints should be sent to Henk G. Schmidt, University of Limburg, Department of Educational 
Development and Research, P.O.B. 616, 6200 MD Maastricht, The Netherlands. 
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Brief Overview of the Medical Curriculum at the University of Lin 
The Maastricht medical curriculum is a 6-year programme. The first 4 years are 
to understanding pathophysiologic mechanisms underlying disease and the fina 
are focused on clinical training. Each year of the curriculum consists of a number o 
"blocks" or units. During each unit, students meet twice a week for a two-hour, snu 
tutorial in which problems are analysed and learning goals are formulated. Thes 
are guided by tutors drawn from the staff whose role it is to facilitate the leaminj 
of the students (Barrows and Tamblyn, 1980). Each unit includes a skill-trair 
gramme in which students are taught diagnostic and therapeutic skills. The 
acquired diagnostic skills are tested in encounters with subjects who simulate p 
In each year, the units are organised around one central theme. In the first year, th 
ways in which the human body adapts to the environment are the focus of study, ] 
units on metabolism, response to infection, regulatory mechanisms and homeost 
second year considers human functions as perception and locomotion and c 
development issues like growth and differentiation. For the third and fourth > 
Faculty has chosen health problems that are crucial in terms of prevalence and s< 
Dutch society. Problems such as pain in the chest, fatigue, and blood loss com 
themes of the units. Electives for a variety of research and clinical experiences 
be chosen as part of the curriculum. 

The curriculum components - problem-based learning, small-group tutorials, and < 
on self-study - distinguish the University of Limburg medical curriculum 
traditional Dutch medical schools with which it is here compared. 

Review of Outcomes 

Student Attitudes toward Instruction 

In a study carried out in Groningen (Bender, 1979) students were asked to wrii 
answering the following question: "What do you think of your instruction?" T 
showed one general trend. These medical students had generally experien 
instruction as boring, irrelevant, and sometimes anxiety provoking. Of course, tl 
exceptions; individual teachers or programmes were genuinely enjoyed and cont 
the students 9 motivation to become doctors. However, in general their medical < 
had not appeared to be an enriching experience. Schmidt and Moust (1981) i 
Bender's study. They randomly selected SO students from the Maastricht probh 
curriculum, 10 from each class 1 , and asked them to write a similar letter at 
experiences. Ninety-two percent of the students responded. The authors extracts 
and negative evaluative remarks from the letters, coded the answers in a m 
categories, and tallied frequencies. They scored 1 47 evaluative comments by the N 
medical students on aspects of their curriculum. Of these, 117 were positive, indie 
the Maastricht medical students had a largely positive attitude toward their cut 

1 At that time only five classes were available for study. 
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A closer look at the student responses is instructive. For example, the tutorial groups in 
addition to being praised could also be a source of discontent One student wrote: 
"During tutorial groups students behave rather disinterestedly or do not react at all. If the 
tutorial does not work the way it should or if a group doesn't click they merely shrug their 
shoulders, even though the tutorial group is the key to the entire study. Your relationship 
with people is what it is all about, now and in the future. And if that is not feasible (not even 
in a tutorial group) then what is? One of my characteristics is that I feel very dependent on 
others to maintain my motivation. In this self-directing approach, the others are my 
stimulus." 

One of the more interesting findings was that first-year students expressed difficulties with 
self-directed learning. They seemed to need at least 6 months to adapt to this new learning 
environment in which they are responsible for what they study and how they study. One 
of them wrote: 

"As we are unfamiliar with a self-directed study programme we are confronted with a 
totally new educational system in the first year. This is strange: because we have to 
discover how to study by ourselves: no certainties; hardly any guidelines, but after a year* s 
hard work it turns out to be less difficult.** 

Opportunities for gaining acceptance within the medical care system and patient care were 
most frequently rated positively. The conclusion should be drawn that early and frequent 
involvement of medical students with a health care provider is of developmental importance 
This is particularly so as a source of motivation. One student wrote: 
"I gained most insights into my studies during my electives with physicians and in the 
hospital. Only then did I realize how little I knew. After each of these rotations I 
experienced a considerable increase in my study activities. For me, the confrontation with 
practice is a very important and positive asset in the curriculum of the Maastricht Medical 
Faculty. Because of this practice you get a very good - be it a rather broad - insight into 
the health care system. It gives you a good impression of what you are studying.** 
These representative qualitative descriptions provide a candid and generally positive 
reaction to the Maastricht curriculum by a random sample of students. 

Study Load 

In investigating study load, Weggeman and Moen ( 1 982) adopted a skeptical point of view 
- they asked whether the positive student attitude, expressed in the Schmidt and Moust 
study, was merely an indication that students were satisfied with the Maastricht curriculum 
because it did not demand too much academic effort. Weggeman and Moen asked: "Does 
an emphasis on independent learning with a de-emphasis on summati ve assessment merely 
produce easygoing students**? To explore this question, they used a comparative approach 
and surveyed the study load of first and fourth year Maastricht medical students during a 
randomly selected week of their respective programme. The authors then compared the 
study load estimates with those from various other Dutch university curricula. They found 
an average study load per week of 34.5 hours for first year students and 39.3 hours per week 
for fourth year students at Maastricht. Students in five natural science oriented program- 
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mes, ranging from biology to medicine, at other universities in The Netherlands spent on 
the average 27.8 hours a week on their study. The results of this comparison indicate that 
there is no reason to suppose that emphasis on independent learning and absence of 
pressure from summative examinations create a learning environment that breeds students 
who put less effort into their studies. 

Years of S'udy and Dropouts 

Post, De Graaff and Drop (1986) investigated the relative efficiency of the Maastricht 
curriculum in terms of producing graduates and limiting dropouts. These are important 
data, as the mean dropout rate at Dutch universities in the early 1 980s was no less than 52% . 
Duratior of study and dropout rate of the first five classes at Maastricht were compared 
with statistics from other Dutch medical schools. The results clearly demonstrated that the 
efficiency of the Maastricht medical school exceeded that of the other Dutch metf' 
schools. The majority of the Maastricht students graduated within 6 and ah . u js. 
Median study duration at the other medical schools ranged from 7 to over 8 yean. The 
percentage of students who graduated of was 8^% for Maastricht and ranged from 64% to 
71% for the other medical schools. The Maastricht curriculum thus produced relatively 
more graduates in less time. As there has been no differential entrance selection of students 
to the various medical schools, the most likely explanation for the higher efficiency of the 
Maastricht curriculum would appear to be the motivating effect of problem-based 
learning. 

Achievement 

The dropout rates are described in a positive light as an achievement of the medical 
curriculum. However, when the data are viewed from a more critical perspective, the 
Maastricht medical 

curriculum could perhaps graduate students who might not have succeeded in other 
schools. To answer this question, Bender from the University of Groningen, and his 
associates from several other medical schools ( 1 984) asked: 'To what extent is performance 
of students in the domain of medical knowledge in a problem based curriculum comparable 
with that of students in a traditional school?" The investigators compared the achievement 
of toe Maastricht medical student with that of students in two traditional medical curricula 
and a reference group of graduates from various medical schools. For each group in the 
comparison, all levels of medical expertise were represented and all groups from first year 
to the sixth ye jt took the same test. The test administered consisted of 200 items, a random 
sample from a large item bank covering medical knowledge as a whole: Clinical, 
biomedical and behavioural. The main conclusion drawn by Bender et al ( 1 984) was that, 
whatever the differences, they were small and insignificant at all levels of expertise. Their 
findings, then, do net support the assertion that lower dropout rates are associated with 
poorer performance and less critical scrutiny by :he medical school at Maastricht. 
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Integration of Knowledge and Clinical Reasoning 

Integration of knowledge is an important topic in advanced learning. Flexible application 
in solving difficult problems can only be expected when knowledge bases are integrated. 
Problem-based curricula, while focusing on the application of "old*' knowledge in new 
situations, are said to encourage the integration of knowledge from different domains (e.g. 
biomedical and clinical knowledge), while traditional curricula seem to assume that 
integration occurs automatically. 

In order to investigate the effects of type of curriculum on integration of knowledge, 
Boshuizen and Schmidt (1990) compared the performances of students from two medical 
schools, one problem-based and one conventional. These (preclinical) students were asked 
to explain how a specific metabolic deficiency could be related with a specific disease. 
Knowledge of biochemistry and internal medicine had to be integrated and applied. In 
addition to these students, two groups of experts, biochemists and internists, were 
incorporated in the study. Students from the problem-based curriculum and the biochemists 
appeared to take an analytical approach to the problem by first exploring the biochemical 
aspects of the problem, and subsequently linking them to clinical aspects. Students in the 
conventional curriculum and internists tended toward a more memory-based approach. 
They searched their memories in order to find a direct answer to the question. This latter 
strategy, however, resulted significantly less accurate responses and more frequent 
failures by the students in the conventional programme. These results suggest that students 
in a problem based curriculum integrate their knowledge better with resultant more 
accurate reasoning than students in a traditional curriculum. Whether better integration 
and more accurate reasoning translate into better diagnostic hypotheses remains to be 
explored. 

Conclusion 

The studies presented here have presented global comparisons of curricula at Dutch 
universities The studies are suggestive, but their quasi-experimental nature does not allow 
us to conclude that the differences are purely the result of differences in the curricula. The 
studies were outcome-orientated and all were at some risk of being unduly influenced by 
shortcomings of sampling. However,all the studies dealt with core issues of the discussion 
whether problem-based learning and related innovations really provide a viable alternative 
to conventionaTbducation. These issues included performance in achievement tests, 
clinical reasoning, efficiency and ef fectivity of problem-based programmes, time spent on 
study, and attitudes toward instruction. The data suggest that there is no particular reason 
to believe that problem-based curricula provide i nferior training for medical students as 
compared with conventional curricula. The data in this paper clearly demonstrate some 
"bottom-line" results: medical students in a problem-based curriculum do not spend less 
time on their studies, and their achievement is comparable to that of students of traditional 
schools. Furthermore, problem-based curricula do appear to provide a friendlier and more 
inviting educational climate; the response of students in their evaluation of instruction 
clearly demonstrated this influence. Such an educational climate facilitates the emergence 
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of positive attitudes toward study and may be responsible for the lower dropout rate. There 
is also some evidence that students in a problem-based curriculum integrate knowledge 
from different domains better and acquire superior reasoning skills. Finally, there is 
growing evidence that problem-based learning approaches and problem analysis in a 
small-group tutorial ma> be an effective facilitator of the processing and retention of new 
information (Schmidt, De Voider, De Grave, Moust, & Patel, 1989). 
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Introspection at the Faculty of Medicine of the University of 
Limburg 

G.D. Majoor 
University of Limburg 
Maastricht, The Netherlands 

Abstract 

The Faculty of Medicine of the University of Limburg in Maastricht has implemented 
problem-based learning from its inception in 1974. On the occasion of a visitation by a 
National Review Committee in 1991, the Faculty had to prepare a self study. Among other 
data, that report contained a survey of the Faculty's strenghts and weaknesses. The most 
wvortant of these are reviewed together with the remedies that are planned to correct the 
dejects. 

Introduction 

In response to a request by the Ministry of Education of the Netherlands, the Dutch Society 
of Collaborating Universities (VSNU) has organised a series of site visits to universities 
since 1991 in order to assess the quality of their faculties' education. Among the first to 
be visited were the Faculties of Medicine and Health Sciences. Each school was requested 
to prepare a *self-study\ in order to provide advance information for the external review 
committee. The present paper describes the current Maastricht curriculum, and more 
specifically some of its strenghts and weaknesses as identified in the self-study. The flaws 
will be discussed with the remedies that have been planned to correct them. 

Educational starting points and curriculum design 

The educational philosophy for the Faculty was originally laid down in a paper published 
in 1972. The major elements, which were distinct from those applied by the other Dutch 
schools at that time, were: 

- student-centred, self-directed learning, 

- multidiscipl inary education, 

- early introduction of students to health care settings, 

- equal attention to extra- and intramural health care in the curriculum 

- attention to the development of skills and attitudes, and 

- emphasis on formative rather than summativc assessment. 

Requests for reprints should be sent to Gerard D. Majoor, University of Limburg, Faculty of Medicine, Office 
for International Relations, R0.B> 616, 6200 MO Maastricht, The Netherlands. 



200 



These starting points are still valid in the Faculty and some of them have even been adopted 
by some of the other Dutch medical schools. 



To implement the philosophy the Faculty introduced from its very inception problem- 
based learning and small group tutorials as the main educational setting for the first phase 
of its curriculum (Years 1-4). Each yearisdivided into 6-week multidisciplinaiyeducatioiud 
units or 'blocks' . The emphasis in these blocks shhti from basic sciences in Years 1 and 
2, and normal human development in Year 2, to major categories of complaints as 
encountered in General Practice in Years 3 and 4 (Table I). A training programme for 
diagnostic, therapeutic, laboratory and communication skills runs throughout this phase of 
the curriculum (Van Dalen, 1 990). Introductions to health care settings commence as early 
as Year 1 . Each block ends with a formative block test. The students* progress is assessed 
four times in each year, when all students sit a summative test at the level of a Final 
Examination called the Progress Test (Van der Vleuien & Verwijnen, 1990). In the last 
phase of the curriculum (Years 5-6) the students undertake clerkships in various hospital 
departments, in general practices and in ambulatory, clinical, and mental health care 
settings. Starting in Year 2 students are offered electives; in total these amount to almost 
one year in the six year curriculum. 



Organisation of Education 

The departure from a curriculum that is organised by departments caused the responsibility 
for the educational programme to shift to the central Faculty level (Bouhuijs, 1990). An 
outline of this central organisational structure is presented in Table II. In addition to this 
core organisation there are quite a number of ad hoc committees to address current issues. 
For instance, a committee for the management of the renewal of the first phase curriculum 
has been in operation for several years. These ad hoc committees usually report to the 
Education Committee. 



Does Bureaucracy Fossilise the Faculty? 

Even a first glame the organisation of education (Table II) may give the impression of a 
cumbersome bureaucracy. Clearly, it is not possible, in a structure of this nature, for a new 
idea to become incorporated into the curriculum within a matter of a week or two. To cite 
an example, a major decision to restructure the curriculum took some four years of in-depth 
discussion before the change was implemented (M ajoor, 1990). 
Although this tsructure may constitute a drawback, in that such a centralised organisation 
inhibits fast response, the structure guarantees democratic decision-making and a fair 
chance for students to have an input into the programme. This extensive organogramme 
and the number of academics involved are sometimes used to demonstrate how expensive 
the Maastricht School must be, when compared with other Medical Faculties. However, 
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Table I.Organisation of education at the Faculty of Medicine 



Decisions on general outlines 
General management 



Strategic management 
& Quality assessment 



Operational management 



Curriculum implementation 
A. Educational units 
(blockSyCkrkships) 



Faculty Council (8 staff, 4 students) 

Faculty Board, 
Vice-Dean for Education 

Education Committee 
Elected; 6 staff, 6 students 
Vice-Dean for Education (advisor) 
Head Education Office (advisor)] 

Coordinating Committee of 

Education: 

Chairperson plus: 

Chairperson Education Committee 

Year Coordinators (5) 

Head Skills Laboratory 

Chairperson Project Group Electives 

Students (2) 

Vice-Dean for Educ. (advisor) 
Head Education Office (advisor) 



Year Planning Groups 
for Years 1,2,3 A and 5/6; 
each consisting of the Year 
Coordinator plus all Block Co- 
ordinators for that year 



Block Planning Groups 
Block coordinator plus 4 staff 
members and 1-2 students 



Tutors 



B. Curriculum support 



C. Student Assessment 



Project Groups for: 
Skills Training 
Practical Medical Training 
Attitude Development 
Student Assessment 
Learning Resources 
Faculty Development 
Programme Evaluation 

Standing Committee for the 

Examinations 

(tenured academics) 



Executive Committee for the 
Examinations 



202 



these antagonists tend to forget that the individual departments in traditional schools also 
spend time and energy on organising their educational activities. Quite apart from any 
argument of cost, department-linked organisations are not likely to meet the quality of 
management attained by this organisational structure for a Faculty as a whole. Another 
potential complication, common to expanding organisations, rests in the allocation of tasks 
and power - in its worst manifestation known as nepotism. At the inception of the school, 
almost all academics held positions in the educational organisation, but as the staff 
expanded there was a tendency to avoid the appointment of newcomers to central roles. 
This was taken for granted by many , until the Faculty adoptedalist of educational activities 
that a member of the academic staff should have fulfilled in order to be eligible for 
promotion. Shortly thereafter discussions on the reform of the curriculum commenced, and 
this opportunity was taken to involve many of the new staff. 

Are All Academics Equally Motivated Towards PBL? 
When the School was established in 1974, such a plethora of tasks had to be fulfilled by 
so few, that individuals had hardly time even to reflect on their motivation. However, as 
the Faculty expanded and the curriculum had largely settled down, there were quite a 
number of staff members who saw education as a rather boring assignment. In their view, 
there was hardly any room for individual initiative in the curriculum. Few lectures and 
practicals were permitted, and tutors were supposed merely to guide the group process but 
not to participate in the students' discussion on the basis of their expertise (Moust et al., 
1990). Moreover, being a tutor twice a week for two hours over a six week period was felt 
to interfere heavily with other commitments that yielded better career prospects, like 
research and health service. 

As mentioned earlier, these complaints were alleviated to some extent by the involvement 
of many staff members in the construction and implementation of the new curriculum 
(Snellen-Balendong, 1990). Furthermore, the concept of the tutor acting strictly as a 
facilitator of the group process was complemented with a role for the tutor as a challenger 
of the student's understanding of the topic in hand. Consequently, departments pertinent 
to the content of a block were identified. Usually, about five departments would be 
acknowledged as "core departments", crucial to the content of a particular block, and 
another four departments as "peripheral". The block coordinator and the planning group 
members were appointed in a revolving fashion from the core departments; tutors were 
recruited exclusively from the core and peripheral departments. This strengthening of the 
linkage of blocks with departments in relation to the content of the block has also facilitated 
the contribution of these departments to the construction of block tests. 
The Faculty Council recently adopted a further measure to promote extrinsic staff 
motivation for education, the review of the evaluation of tutors' function by the students. 
Although standard evaluation forms, to be completed by the students at the end of each 
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block, had for many years contained 10 questions on the performance of the tutor, the 
impact of this part of the evaluation had been quite small (Gijselaers, 19%). For example, 
a tutor, who had been formally registered for a given block by the Office of Education, 
might actually have sent one or several substitutes on a number of occasions. It would then 
be no longer clear to whom the students comments referred. The data on the performance 
of tutors were handled confidentially by the Office for Education; they were usually 
transmitted merely to the tutor and stored by the Office. Only in cases of extreme 
dysfunction would the block coordinator and the Coordinating Committee call the tutor to 
account. The new regulations identify the tutor by name. Evaluation data will be made 
public, and repeated malfunction will be dealt with through discussion with the bodies 
responsible for the implementation of the curriculum and remedial tutor training. Satisfactory 
performance as a tutor will lead tc other roles in the organisation of the curriculum: these 
are mandatory for progress to a tenured academic career. Those tutors who consistently 
perform satisfactorily will regularly be given a small token of appreciation. To further 
strengthen the tutor's role, the Faculty is now reviewing the issue whether tutors should 
also judge the students' performance in the tutorial group (see also below). 

Are all Students doing Weil in PBL? 

The Dutch government system for the allocation of students to the Faculties of Medicine 
excludes selection by individual schools. New students, therefore, exhibit a broad 
spectrum of affinities for the Maastricht educational approach. At the negative end there 
are students who were allocated to Maastricht, although they would have preferred to go 
to a traditional school. Even so, almost all students started the first Year with great 
enthusiasm, but the earliest signs of decreasing motivation began to show in the last block 
of the first Year. Students were given their results on the last summative progress test early 
in that block, and, in the perception of many, the holidays seemed to commence the 
moment they knew that they had passed the first Year, although the block continued for 
several more weeks. On the whole, student attendance in tutorial groups in the first phase 
of the curriculum has represented a problem. Attempts were made to reduce this problem 
by making attendance at tutorial groups mandatory. However, this had to be dropped 
eventually ibr legal reasons. The present strategy is to try to improve the quality of the 
tutorial group sessions. One element of this strategy is to control the quality of the block 
book (which includes the problems to be studied together with lists of the associated 
resources). Block books for the new curriculum were reviewed by a panel of experienced 
members of the Faculty. Data, derived from evaluation by students and tutors were 
scrutinised by the Coordinating Committee. The latter also checked whether weak spots 
in the blocks were adjusted in response to these evaluations. Another action to improve 
student satisfaction with the tutorial group sessions was to pay more attention to the tutor's 
role. As mentioned above, the Faculty is considering the introduction of formal assessment 
by the group tutor, as practised for example at McMaster University Faculty of Health 
Sciences. Originally student assessment was not included iu Maastricht tutorial group 
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sessions, in order to avoid undesirable competition among students. Its introduction in 
some form, however, may be necessary to reduce the problem of low student attendance 
and to reinforce the role of the tutor. 

Do we Need Proof of Excellence? 

In earlier paragraphs, emphasis has been placed on some of the weaknesses and problems 
that have been identified by the Maastricht Faculty of Medicine. This might raise the 
impression that the School is struggling for survival. Fortunately this is not the case. 
Obviously, embarking on experimentation with new approaches in (medical) education 
represents a long road of trial and error. However, thanks to its innovative approach, the 
School has an outstandingly well-organised educational system. The organisational 
structure also makes it possible to document accurately the Faculty' s manpower investment 
in education, and to control financial expenditure. The centralized design and marking of 
the examinations guarantees a consistently high quality of the tests and permits detailed 
feedback to the students on their strengths and weaknesses related to disciplines and organ 
systems. Both student assessment and programme evaluation have yielded a wealth of 
data, and some of this infonn ition has been used in further investigations by the Faculty's 
Department of Education Development and Research. 

However, these organisational strengths cannot answer the question most frequently 
asked: "Are Maastricht graduates superior to those who were educated at the other Dutch 
medical schools?" It should be obvious that no clear-cut answer can be given to this 
question; an instant rejoinder would be: "Superior in what respect?" However, the first 
question has naturally intrigued the Maastricht Faculty of Medicine, and there are some 
data that can be cited. Four studies have been undertaken to compare the medical 
knowledge of Maastricht students to that of students of some of the other Dutch schools.* 
In all but one study no statistically significant differences were demonstrable, but all 
studies invariably suffered from methodological flaws which were hard to avoid. With this 
reservation in mind, it was concluded that, although these studies did not indicate 
superiority of Maastricht students in the cognitive domain, they also found no inferiority 
in students educated by this Faculty. Another parameter was the efficiency Of the 
educational programme. A survey performed nine years after the start of the Maastricht 
curriculum provided evidence that favoured the Maastricht educational approach. Both 
students* mean study time and drop-out rates compared very favourably with those of the 
other Dutch schools (Verwijnen, 1990). Once more a warning applies to the interpretation 
of these data; the first classes in Maastricht consisted of only 50 students who, therefore, 
enjoyed a very low student-to-staff ratio. In the meantime, these figures for Maastricht and 
the other Dutch schools have come closer to each other, although the Maastricht figures 
are still among the best in The Netherlands. 

Those wanting to "prove" superiority of graduates educated in innovative curricula should 
seriously address the question whether it is imperative to prove better performance. Let us 
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suppose that studies like those quoted above had actually indicated an inferiority in 
Maastricht students in the cognitive domain, or that the Maastricht Faculty had been less 
efficient in student output than the other Dutch medical schools. These, in themselves 
worrying, data would by no means preclude that Maastricht graduates might be better in 
communicating with patients, in arriving at the correct diagnosis, or in keeping abreast 
with new information. Although attempts to demonstrate differences between strategies 
in medical education should proceed, it would be wise to take into account that it may never 
be possible to give a definitive scientific answer to "what's best?" 

Epilogue 

The Faculty at Maastricht was visited by the National Review Committee in October, 
199 1 . After it has visited all medical schools in the Netherlands the Committee is expected 
to publish a report on its findings; this report will appear in the Spring of 1992. However, 
at theconclusion of the visitation at Maastricht, the Chairman summarized the Committee's 
first impressions. Apart from a number of critical comments, partly referring to aspects 
also discussed in this paper, a comparable number of compliments was offered. The 
Faculty was pleased to note in one of the Chairman's final statements: "on the whole, our 
impression really is that education here is excellent; we are very satisfied with that". The 
critical points identified by the Faculty and the Review Committee are currently being 
addressed in order to try to effect further improvements. 
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Evaluation of Thtor Effectiveness: A Review of the Literature 



J.G. Wakefield, D. Keane and J. Hay 
McMaster University 
Hamilton, Ontario, Canada 

Abstract 

The assessment of the performance of tutors is an essential aspect of programme 
evaluation in those curricula where problem-based learning ( PBL) is a significant feature. 
However, few published evaluation tools contain categories which are specific for the 
evaluation of tutors in problem-based learning curricula. 

A combination of search strategies was vital in this project An extensive review of the 
literature identified only one article that was directly relevant to students' evaluation of 
tutors 9 effectiveness. However, personal contact with individuals working in other schools 
with PBL curricula resulted in additional information that was very helpful in validating 
the content of a new evaluation form. 

Key tasks, identified for tutors, are summarised, with a brief list of references. 



Introduction 

Given the integral role of the tutor in small group problem-based learning (PBL), 
evaluation of the performance of tutors is an essential aspect of programme evaluation in 
curricula with a significant PBL component. Traditional tools to evaluate academics tend 
to measure attributes of more traditional "teaching" and contain few categories which are 
appropriate for the evaluation of tutors (Irby, 1983;Rippey, 1981). Anumber of papers do 
describe the roles, requisite skills, and basic attitudes of the tutor who is involved in small* 
group PBL(e.g., Barrows andTamblyn, 1980; Maxwell and Wilkerson, 1990;Grand*Maison 
and Des Marchais, 1991). However, these neither describe forms nor report studies that 
evaluate the performance of such tutors. 

Arising from the need for an effective tutor evaluation system in the new Occupational 
Therapy and Physiotherapy Programmes at McMaster University, we reexamined the tutor 
evaluation forms currently in use in the medical and nursing programmes, and we 
conducted several reviews of the literature during 1 990 and 1 99 1 . These were two key steps 
in a current study of a new form for evaluation of tutors by students. 

Requests for reprints should be sent to J.G. Wakefield, First Place Family Medical Centre, 106-350 King Street 
East, Hamilton, Ontario L8N 3Y3, Canada. 
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Method 

We realised from previous work that the number of published studies on the evaluation of 
small group tutors was limited. We, therefore conducted generally equivalent systematic 
searches forrelevant English language literature in four major CD-ROM indexes. We 
defined "relevant" to include literature reviews, empirically-based evaluation studies, 
expert opinion, and "we-are-doing-this" reports addressing the problem of how to assess 
the quantity and quality of teaching provided by faculty tutors in small-group, preferably 
problem-based, learning settings at university level. 

We searched MEDLINE 1966-1991 for citations containing any variation on the word 
"tutor" (e.g., tutored, tutorial, tutoring, tutors) in their title or abstract and "education" of 
"faculty" (including their subdivisions "dental", "graduate", "medical", "nursing" and 
"pharmacy) but not "computer-assisted instruction" or "computers" (and their subdivisions) 
among major or minor MESH headings. Other strategies, including searches based on 
synonyms for "tutor" , were found to be unsuitable. 

We then searched the Cumulative Index to Nursing and Allied Health Literature 
(CINAHL) 1983-1991 for citations containing any variation on the word "tutor" in their 
title or abstract and "faculty, nursing" or "faculty, allied health" but not "computer-assisted 
instruction" among their major or minor CINAHL headings. Alternative strategies, 
including restricting items to those containing "promotion and tenure" among their 
CINAHL headings proved unproductive or too restrictive. 

We also searched the PsyLIT Abstracts, 1983-1991 for citations containing "college 
teacher" and either "personnel evaluation" or "teaching fellows" in their abstracts or any 
variation on "tutor" in their title, abstract, or "key phr?.ses". 

Finally, we searched the Educational Resources Information Center (ERIC), 1980-1991 
for citations with ERIC descriptors "tutor" and "post-secondary education" or "higher 
education". 

Following each search, we evaluated identified items by title, abstract and major and minor 
headings. We read those that seemed relevant and possibly informative. We also examined 
references in literature previously known to us, in order to identify books and other 
possibly relevant resources not identified by the above searches. 

In addition to the formal review of published literature, we made personal contact with 
individuals working in problem-based schools in North America and known personally to 
members of our research group. A letter, outlining our task and our request, was sent to 
individuals at five medical schools. The response was very gratifying, as all but one replied. 
All respondents forwarded copies of the forms currently in use in their institutions, and two 
sent additional, relevant materials. 
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Results 

Our searches identified over 200 articles. The majority were found using Medline (174). 
Thirty-two additional articles were added from CINAHL, and six from PsyLIT. No articles 
of value were added from ERIC. An initial screening of the articles by one of us limited 
the number for further review to 28. A second screening quickly reduced the number to 
nine, based on information in the printed abstracts. One of the problems in the search came 
quickly to light in reading the abstract of a paper from Leeds in the U.K. (Stanley & Wright, 
1981). These authors defined "tutorial-type teaching" as "one student to one doctor". All 
but two of the final nine articles used "tutor" or "tutorial" in this sense. 

However, these searches did yield one paper that described a form for students' evaluation 
of tutor effectiveness. It came from investigators in the School of Medicine at the 
University of Newcastle in Australia. In addition to providing an outline of the form, it also 
presented the results of a study using the form. 

This paper, by G.I. Feletti, E. Doyle, A. Petrovec and R. Sanson-Fisher, is entitled Medical 
students 9 evaluation of tutors in a group-learning curriculum (Medical Education, 1982, 
16, 319-325). Their study "focused on students' evaluation of tutors' effectiveness during 
an academic term, using a nineteen-item questionnaire." It took place within an integrated 
problem-based learning medical course. They identified four major factors in the tutors' 
behaviour which had "importance in the rating process: the ability to care for students, a 
knowledge of course structure and teaching staff philosophy, the ability to encourage 
independent thinking in students, and a knowledge of the specific medical problems being 
studied." 

This paper reported many similarities to our own studies. The result of the factor analysis 
is one example. Based on a questionnaire of 16 items, our initial study had identified three 
discretefactors - which were labelled "benevolent authority", "challenger of development", 
and "course objectives" (Hay, 1991). 

The forms and materials from PBL schools also support the importance of tutor skills that 
encompass these dimensions. In particular, the response from Sherbrooke was accompanied 
by a copy of the paper, "Validation by Network and Sherbrooke Tutors of Problem-Based 
Learning Tutor Tasks," presented at the Second International Symposium on PBL in 
Yogy akarta, Indonesia (Des Marchais and Chaput, 1 990) This paper both outlined specific 
tasks identified for the tutor's role (based largely on work by Barrows and Schmidt) and 
described the process used in a face validity study of these tasks. The tasks rated as most 
important for tutors by designated schools in the Network were "facilitate the functioning 
of a small group", "favour humanism and autonomy", "manage the PBL method", and 
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"guide the study of specific contents". Other tutor tasks were also cited, and some of these 
were ranked as very important in some PBL schools. 

An additional paper (Gijselaers and Schmidt, 1991) was identified by an interested 
colleague. They describe a project that was part of a system for process evaluation of the 
problem-based educational programme of the Medical Faculty of the University of 
Limburg in Maastricht, The Netherlands. Their student evaluation questionnaire included 
aspects of the performance of tutors, and information is given about the results of both 
factor analysis and determination of interrater reliability. They conclude with a statement 
about the importance of a theory of teaching and learning to "posit the evaluation 
dimensions" in the construction of a rating scale. 

Conclusion 

In summary, the results of an extensive review of the literature identified only one article 
that was relevant to students' evaluation of tutors' effectiveness. 

However, direct contact with individuals working in other schools with PBL curricula 
resulted in additional information that was very helpful in validating the content of our 
evaluation form. 

This type of "multi-method" strategy to identify relevant literature may be extremely 
valuable in projects of this type, especially when a search of the standard literature alone 
contains few relevant articles. 
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Evaluation of the Lecturer: The Ideal and Reality 

Hristina Zhivkova 
Medical Academy 
Sofia, Bulgaria 

Abstract 

This research gave us an opportunity to understand what criteria students use to evaluate 
their lecturers and how lecturers evaluate themselves. These criteria were compared with 
the "ideal" lecturer, as identified by students and teachers. Evaluation by students, and 
self-evaluation by teachers, and comparison with the "ideal" lecturer are not without 
significance for the student-teacher relationship in 'he educational process. 

Introduction 

The research involved 200 students and 200 lecturers from three medical schools in 
Bulgaria. One hundred fourth year students and 100 fifth year students were questioned. 
Students from the senior classes were addressed, because they had a long experience of 
their lecturers during lectures, seminars, practical, and in the clinics. 
Two hundred lecturers from the same three institutions were questioned. They were 
selected randomly. These teachers included lecturers (140) and associate professors (60) 
with experience ranging from 3 to 20 years. One hundred and thirty were clinical teachers 
and 70 were teachers in the basic sciences. 

The aim of the research was to identify the system of values on which the criteria for 
evaluation of the personality of the "ideal" lecturer were based by the students and the 
lecturers. The investigation took place during the 1988-1989 academic year. The research 
was undertaken by the staff of the Paedagogical Department of the Higher Education 
Institute, Sofia. 

Method 

The questionnaire (Appendix) contained three groups of qualities, which could be 
possessed by the "ideal" lecturer: professional, educational and personal qualities. The 
respondents were asked to value these qualities on a scale for low, medium, or high. The 
students were asked to evaluate the qualities for an "ideal" and for their "real" lecturer. The 
lecturers were asked to identify the qualities of the "ideal" lecturer and to evaluate their 
own qualities. The specific qualities were selected on the basis of a professional evaluation 
and after discussions with lecturers who had participated in the course on medical 
education during 1986 to 1988. 

Requests for reprints should be sent to Hristina Zhivkova, Bl. 13, Ap. 52, Zona B- 18, 1309 Sofia, Bulgaria. 



205 



216 



What kind of professional, teaching and personal qualities should the lecturer possess; 
what are the most valuable qualities of a lecturer from the point of view of the students? 
Where do their views differ? To what extent can the students experience and evaluate the 
lecturers? 

The lecturers in the Higher Institutes of Medicine are involved in teaching, patient care and 
research. Students can gain more realistic impressions in respect of the first two, because 
they can observe their teachers in these activities. 

Results 

Professional Qualities 

The criteria of the students and the lecturers were very similar for the professional qualities 
of the "ideal" lecturer. The opinions of the students and lecturers in respect of the "real" 
lecturer demonstrated considerable differences. A few students detected to a high degree 
the ability of their lecturers to accept "with respect the achievements of other experts". The 
number of students who detected to a high degree the quality of Unselfishness" in their 
teachers was smaller. The smallest of all was the number of students who detected to a high 
degree the quality of "humanism" in their lecturers. Hie opinion of the lecturers was rather 
different in terms of their self-evaluation in these three qualities. The most significant 
difference in the evaluations between the students and the lecturers was in terms of the 
criterion "dedication in the discharge of the professional duty". While self-evaluation was 
more favourable, the students* evaluation was more critical. 

A significant difference was demonstrated in the opinions of the students and lecturers for 
the "real" lecturer in terms of most of the qualities. In connection with the indices for 
"professional knowledge, skills and habits" and "empathy", the difference between self- 
evaluation and the students* evaluation was significant at P<0.05. There was a difference 
at P<0.001 for almost all the other qualities. The students were very critical of the 
professional qualities of their teachers; most of the students evaluated the professional 
qualities of their lecturers as "low" and "medium". 

The self-evaluation of the lecturers could also be characterised as critical. This was 
confirmed by comparing data in the evaluation of the "ideal" lecturer with self-evaluation 
by the lecturers. The lecturers estimated highly most of the professional qualities in the 
"ideal" lecturer. At the same time, they were quite self-critical and did not find these 
qualities to a high extent in themselves. 

The students' high expectations of their teachers became apparent when comparing the 
results of their evaluation of the "ideal" lecturer and of their "real" lecturers. The difference 
was not significant only in terms of "ability to work in a health team". In terms of all the 
other indices, the difference was significant at P<0.001 . A large number of the students 
thought that the "real" lecturer should possess to a high a degree "professional experience", 
but only a comparti vely small number of students thought that their real teachers possessed 
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this quality to high extent. The data for "high erudition", "apply the norms of medical 
ethics", "humanism", "empathy", Unselfishness", were very similar to the above results. 

Educational Qualities 

While the criteria of the students and the lecturers for educational qualities of the "ideal" 
lecturers coincided closely* there was an important difference in the view of reality 
between the lecturers and the students. The most striking differences were in the opinions 
of students and lecturers related to "showing paedagogical tact", and "teaching scientific 
knowledge accessibly". 

When the results of the lecturers' responses about the "ideal" lecturer and their self- 
evaluation were examined, essential differences were found. The lecturers were rather 
self-critical and had a low opinion of most of their own educational qualities. 

It was demonstrated that 82.70% of the students valued highly the "possibility for detecting 
the connection of the taught material with the practice" in the "ideal" lecturer, but only a 
few students (16.75%) perceived this quality in their teachers to a high degree. Also, only 
16.75% of the students thought that their actual teachers possessed to a high degree the 
ability to "contribute to the development of the necessary professional skills and habits". 
Furthermore, only a small number of the students (9.75%) thought that their teachers 
possessed to a high degree the ability to assess impartially. 

Only 20% of the students thought that their teachers possessed the ability "to teach 
scientific knowledge in a simple language", while they valued this ability highly in the 
"ideal" lecturer. Only 1 6.75% of the students thought that their teachers possessed to a high 
degree "wealth and culture of speech". A comparatively small number of the students 
(2 1 .62%) thought that their teachers possessed to a high degree the skill "to direct attention 
to principles". A very small proportion of the students (8.65%) thought that their teachers 
"created conditions for discussion". A large number of the students (80%) thought that the 
"ideal" lecturer should possess the ability "to illustrate the educational content well", while 
only 21.08% of the students thought that their teachers possessed this ability to a high 
degree. Only 9.73% of the students thought that their teachers possessed the ability "to take 
into consideration the opinion of the students", while 71.89% of the students thought that 
teachers should possess this ability to a high degree. The number of the students who 
thought that their teachers showed great paedagogical tact was not large (9.19%), while 
many students (69.72%) thought that the "ideal" lecturer should possess this quality to a 
high degree. 

There was thus a big discrepancy between the students' expectation and their view of 
teachers with regard to these educational competences. 

Personal Qualities 

Here, as with the other two groups of qualities, the same tendency could be observed. The 
students and the lecturers concluded that the "ideal" lecturer should possess to a high extent 
the qualities listed in the respective part of the questionnaire. For more than ten indices the 
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difference was not significant. There was a significant difference for "a sense of 
responsibility**, "professional ethics", "a requirement of the principles", "self-evaluation", 
"fantasy", "civil position", Watchfulness", "accuracy" and "exactness". However, for the 
qualities: "a sense of responsiblity", "scientific and professional ethics", "stability to 
external influences" and "charm and personal attractiveness" the difference between the 
evaluation by the students and the lecturers was not reliable. For the qualities: "efficiency", 
watchfulness", spiritual balance", "self-watchfulness", "fantasy", "a sense of humour", 
"self-evaluation" and "quickness of mind", the difference was significant at P<0.05. For 
all other qualities a reliable difference at P<0.00i was established. 
The opinions of the lecturers about the "real" and the "ideal" lecturer were significantly 
different. The difference between the perceived "ideal" and "real" lecturer for "a sense of 
responsibility", "self-evaluation" and "fantasy" was unreliable at a level of significance 
P>0.005. For all the other qualities the difference between the opinion of the lecturers for 
the "ideal" and the "real" lecturer were reliable. 

Conclusions 

The students did establish criteria for evaluation of their lecturers. On the basis of these 
criteria they formed expectations. The students used the same criteria to establish their 
"ideal" and to evaluate the "real" lecturers. There was a big difference between what the 
students expect and value highly in the "ideal" lecturer, and what they find to a high degree 
in their real teachers. Significant differences were also established between teachers* 
evaluation of the ideal and self-evaluation. 

It is neither possible nor necessary to equate the perceptions of the students with those of 
the lecturers. However, it is necessary for lecturers to know what the students expect of 
them. 

The students* evaluation of their teachers is significant in so far as it builds attitudes 
towards the lecturers and thus affects the students* confidence in, and respect for their 
teachers. It determines also the relationship between lecturer and student, as well as the 
attitude towards the educational process and the subject to be studied. 
It should be noted that the students* evaluation is quite subjective. They tend to evaluate 
from the point of view of their interests and needs. Even so, they have their own criteria, 
and lecturers should be aware of the criteria by which they are being evaluated by their 
students. 
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APPENDIX 

The questionnaire and accompanying letter 

Dear lecturer (student), 

You are invited to participate in this research, which aims to establish your 
opinion on the qualities of the lecturer in your medical school. We rely on your 
conscientiousness in completing the questionnaire. 

Please identify: 

1. Which of the qualities given below make up the "ideal" lecturer, and to what 
extent (low, medium, high) these qualities should be demonstrated. 

2. To what extent do you, as a lecturer, possess these qualities? Please evaluate 
yourself. As a student, to what extent do you think your lecturers possess 
these qualities? 
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Degree 



Professional qualities 



a) low medium high b)low medium high 



1. Professional experience 

2. Professional knowledge, 
skills, habits 

3. High erudition 

4. Skill of communication with 
patients and their relatives 

5. Good professional self- 
confidence 

6. Controls the norms of 
medical ethics 

7. Respects the achievements 
of other specialists 

8. Knows how to stand up for 
own opinion 

9. Ability to work in a health 
team 

10. Ability to act as a specialist 

11. Humanism 

12. Dedication to professional 
duty 

13. Compassion 

14. Empathy 

15. Unselfish 



2 
2 

2 

2 

2 

2 

2 

2 
2 
2 

2 
2 
2 
2 



3 
3 

3 

3 

3 

3 

3 

3 
3 
3 

3 
3 
3 
3 



2 3 
2 3 



2 
2 
2 
2 
2 



2 
2 
2 

2 
2 
2 
2 



3 

3 

3 

3 

3 
3 
3 

3 
3 
3 
3 
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Educational a) low 


medium high 


b)low medium 


high 


1 . Teaches scientific knowledge 










in a simple language 1 


2 


3 


1 2 


3 


2. Demonstrates the material 










in a logical sequent 1 


2 


3 


1 2 


3 


3. Interprets the scientific 










knowledge in depth 1 


2 


3 


1 2 


3 


4. Can direct attention to the 










essentials of the information 1 


2 


3 


1 2 


3 


5. Creates conditions for 










discussion 1 


2 


3 


1 2 


3 


6. Has respect for the opinions 










of the students 1 




3 


1 2 


3 


7. Exhibits a wealth and culture 










of speech 1 


2 


3 


1 2 


3 


8. Good demonstration of the 










educational content 1 


2 


3 


1 2 


3 


9. Reveals the relationship of the 










taught material with actual 










practice 1 


2 


3 


1 2 


3 


10. Helps to build skills for self- 










education 1 


2 


3 


1 2 


3 


1 1 . Helps in the development of 










professional skills and habits 1 


2 


3 


1 2 


3 


12. Treats students as individuals 1 


2 


3 


1 2 


3 


13. Demonstrates educational 










sensitivity 1 


2 


3 


1 2 


3 


14. Assesses students objectively 










and impartially 1 


2 


3 


1 2 


3 


IS. Creates a cheerful and active 










ClllllUoLFllVl w 1 


2 


3 




% 


16. Instills respect for the achieve- 










ments of our own and foreign 










scientists 1 


2 


3 


1 2 


3 
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Personal 



a) low medium high b)Iow medium high 



1 . A sense of responsibility 

2. Scientific and professional 
ethics 

3. Aware of the requirements 
of social behaviour 

4. Stability in the face of external 
influences 

5. Health and spiritual balance 

6. Charm and personal attractive- 
ness 

7. Feeling of humour 

8. Efficiency 

9. Imagination, creativity 

10. Initiative 

11. Self-criticism 

12. Civil position 

13. Diligence and capacity for 
work 

14. Awareness of others 

15. Self-awareness 

16. Quickness of mind 

17. Communication 

18. Exactness and accuracy 

19. Justice 



2 
2 

2 
2 
2 
2 
2 
2 
2 

2 
2 
2 
2 
2 
2 
2 



3 
3 

3 
3 
3 
3 
3 
3 
3 

3 
3 
3 
3 
3 
3 
3 



2 
2 

2 
2 
2 
2 
2 
2 
2 

2 
2 
2 
2 
2 
2 
2 



3 
3 

3 
3 
3 
3 
3 
3 
3 

3 
3 
3 
3 
3 
3 
3 
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INNOVATION AND CHANGE 

The Role of Evaluation in the Development of a Problem-Based 
Learning Programme within a Traditional School of Medicine 

Margaret E. Lewis, Ann Buckley, Margaret Kong and Graham W. Mellsop 
University of Otago 
Wellington, New Zealand 

Abstract 

The gradual development of a problem-based, student-centred clinical sciences course in 
the first clinical year of a traditional school of medicine is described. After four years, the 
course replaced the existing lecture-based, didactic clinical sciences programme. 
Throughout, the impact of the pilot problem-based programme on students' approach to 
learning and their performance in examinations has been compared with that of the 
lecture-based programme. The results of this evaluation are described and the role it has 
pi xyed in the process of change and ongoing development is discussed. 

Introduction 

The literature on the empirical evaluation of undergraduate medical courses is small. The 
large number of relevant but not easily controllable variables may have deterred many 
investigators; others may have been convinced they already knew the answers. The 
introduction of innovative teaching and learning programmes into traditional medical 
schools is itself such an overwhelming task that time-consuming and difficult evaluation 
is often neglected. 

That the wholesale switch from traditional to problem-based curricula is possible has 
recently been shown by the Sherbrooke Medical School (Des Marchais, 1 99 1 ). In contrast 
to this, we describe the gradual introduction of a problem-based clinical sciences course 
(PBCSC). Throughout this development we have attempted to evaluate the impact that the 
changes have had on the students 9 approach to learning and on their academic performance. 

The primary aim of this innovation was that the PBCSC should ultimately replace the 
didactic lecture-based clinical sciences course (LBC) during the fist year of clinical 
medicine at the Wellington School of Medicine. This is the 4th year of the medical 
undergraduate programme of the University of Otago in New Zealand. The goal of 
replacing the LBC was achieved in 1991 after piloting the PBCSC for four years. A further 
aim, that the PBCSC should be integrated into the relevant clinical attachments, for 

Request for reprints should be sent to Margaret E. Lewis, Wellington School of Medicine, Department of 
Paediatrics, Wellington, New Zealand. 
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example predominantly respiratory problems were studied in the general medical run, has 
only been partially achieved. 

The purpose of this paper is to outline the development of this PBCSC, to describe the 
approaches that were adopted to evaluate it, and to report the results of this evaluation. The 
discussion will focus on the role that this evaluation has played in the process of change, 
in the ongoing development of the PBCSC, and in the undergraduate programme as a 
whole. 

Development of the PBCSC 
Background 

The preclinical years (2nd and 3rd) are spent in Dunedin where problem-based learning 
(PBL) has been used increasingly as a teaching method since 1986. This innovation 
followed two Faculty of Medicine residential workshops and a National Conference on the 
Role of the Doctor in New Zealand (Summary Report, 1985). Until recently, PBL was an 
intradepartmental initiative using differing approaches and specialist tutors (Heath, 1 988; 
Schwartz, 1 989). On completion of 3rd year, students elect to go to one of the three clinical 
schools in Dunedin, Christchurch or Wellington (Fig. 1). 



The Wellington Problem-Based Learning Programme 
The stimulus for change came initially from a small group of academics whose knowledge 
and experience of student centred, small group, problem-based learning, as developed 
initially at McMaster, had convinced them of it* s almost unique ability to promote the 
development of many skills required by students after graduation. Increasing student 
enthusiasm for PBL, as indicated by the numbers wishing to take part in the PBCSC, was 
an important stimulus for its ongoing development (Table I). 

Table II outlines the important stages in the development of the PBCSC. An attempt to 
incorporate PBL into the LBC in 1986 was unsuccessful despite the students' previous 
experience of PBL. This was an isolated "experiment" in the midst of a didactic 
programme, where transmission of a large amount of knowledge from teacher to student 
was the primary goal. Student groups were randomly selected from the class, and most 
tutors had little experience of PBL. 

The first two years of the pilot ( 1 987-8) took the form of a limited PBCSC which was added 
on to the LBC, using non-specialist tutors and Faculty staff nominated as resource persons. 
The report to the curriculum committee in 1987 after the fist year concluded that the 
strengths of student-centred PBL were seriously compromised, if students were also 
expected to attend a teacher-centred didactic course. This effect was magnified by the 
presence of an end of year assessment which rewarded recall of facts rather than ability to 
apply them in the clinical setting. Due to time constraints, no major changes were made 
to the PBCSC in 1988. 
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Table I. Student participation in the PBSC 1987-1991 



Year Student Numbers 

PBCSC. LBC. Total 



1987 


11 


39 


50 


1988 


16 


44 


60 


1989 


13 


46 


59 


1990 


18* 


39 


57 


1991 


60 


0 


60 



* A total of 39 students opted for the PBCSC. A ballot had to be held to select 18, the 
maximum possible, given the available resources. 




June 1 986: Unsuccessful incorporation of PBL into the Respiratory section of the 

LBC. 

Dec. 1986: Approval of 2hrs/weck self-directed learning time within the LBC to 

mount a pilot PBCSC. 

March 1987- 1st Pilot PBCSC - in addition to the LBC. 
November 1988: 5 problems studied over 21 weeks. 

Dec. 1988: Approval of a PBCSC in parallel with the LBC, with a separate 

pathology programme. Approval to change 4th year examination to an 
OSCE. 

March 1989- 2nd Pilot PBCSC - mounted in parallel with the CSC*, 23 problems 
November 1990: studied over 29 weeks. 

Early development of a self-assessment programme (SAP). 

July 1990: Approval that in 1991 the LBC should be replaced by the PBCSC with 

a separate pathology programme. Continued development of the SAP. 

March 1991 : Whole class PBCSC with 2hr/wk lecture programme. 

* CSC = Clinical Science Course 



• L 215 



226 




Stewartps 170 ° 175 0 
Island^ «**t •1st 



Figure 1 . Location of the three clinical schools of the Faculty of Medicine, University of 
Otago 

The nature and results of the formal evaluation of the 1987/88 PBCSC were reported 
brief ; (Mellsop et al., 1989) and are further described below. 

In October 1 988, a small group of 3rd year students indicated that they would prefer to learn 
their 4th year clinical sciences in a problem-based rather than a lecture-based programme. 
A proposal to mount a PBCSC in parallel with the LBC in 1989, with all students attending 
a separate pathology programme, was approved by the Wellington Faculty in December 
1988. Fifteen students volunteered to take part in this parallel PBCSC. 
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Discussion with the LBC subspecialty working parties, and analysis of the content of the 
1988 lecture programme led to the definition of a "core clinical sciences curriculum". On 
this basis eighteen new patient problems (PP) were selected. The design of the existing five 
PP's was modifies in the light of the 1987/88 experience, and this structure was adopted 
for all the PP's. 

As before, in addition to the case material, each PP included specific learning objectives 
and recommended learning resources for use by the students. This parallel programme ran 
for two years (1989-90). Every attempt was made to integrate the PP's to be studied with 
the content of the students' current clinical attachment. 

A self Assessment Programme has been developed alongside the tutorial programme since 
1989. Various formats have been used for the self-assessment modules. The commonest 
is the Modified Essay Question. Almost all these modules are problem-based. Cases have 
been selected and the modules designed to reinforce and expand the students' knowledge 
and understanding of common and important clinical problems. As with the PP's, there are 
specific learning objectives and recommended learning resources for each module, and the 
students' also have access to the model answers and to nominated resource persons. This 
programme is still in it's development phase. 

Approval was obtained in December 1988 to change the format of the end of 4th year 
summative assessment from a short answer, factual recall format (primarily testing the 
clinical sciences) to an Objective Structured Clinical Examination (OSCE) which would 
assess both the clinical sciences programmes (CSP) and the clinical attachments. Students 
were to sit a formative mid-year examination to familiarise themselves with the format. 
Pathology would be assessed separately, with a written knowledge-based examination and 
a practical. 

In July 1990 the Faculty in Wellington approved a proposal that the LBC should be 
replaced by a whole-class PBCSC in 1991. This was to be administered in a similar way 
to the 1989/90 parallel PBCSC, except that it would include a "core concept" clinical 
sciences lecture programme with a maximum of 2 lectures a week. Pathology would 
remain a separate programme. The formal evaluation of the 1991 whole-class PBCSC has 
yet to be completed. 

Evaluation of the Problem-Baseu Clinical Sciences Course 

Throughout the four year pilot phase, there have been two major components to the 
evaluation, measurement of the students' approach to learning and of their academic 
achievement, both before and on completion of the programme. 

Evaluation of Approach to Learning 

The tool selected was the Adelaide Diagnostic Learning Inventory for Medical Students 
(ADLIMS) developed by Newble ct al. (1988). This had been administered in 1986 to 2nd 
and 3rd Year students in Dunedin, and we used these data to reexamine the factor structure 
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using the FACTOREP procedure (Welch et aL, 1990). As a result, instead of the 4 factor 
structure, described by Newble, three subscales were defined with acceptable replicability 
across the two medical student populations, and 14 of the 53 ADLEMS items were 
discarded. Three approaches to learning were identified: 

Subscale 1: Low motivation to study and distraction arising from social activities. 
Subscale 2: Worry arising from high achievement expectations that are hampered by 
superficial learning or disorganised study habit. 

Subscale 3: Learning characterised by visualisation, inquisitiveness, reflection, hypothesis 
generation and the linking of concepts across subjects studied. 

Subscale 3 resembles the "deep approach" discussed by Newble and Entwistle ( 1 986). The 
other two subscales represent unfavourable approaches to learning. Subscale 2 includes 
the "superficial approach" which they also discuss, but the "strategic approach" they 
describe does not emerge as a distinct entity. 

We have used this modified version of ADLIMS (Newble et al., 1988) to survey medical 
student populations; not as a "diagnostic" tool to identify individual students whose 
approach to learning is predictive of unsatisfactory performance, the purpose for which 
ADLIMS had been developed. Our aim has been to discriminate confidently between 
different populations of students as regards favourable (subscale 3) versus unfavourable 
(subscales 1 and 2) approaches to learning. In the longer term, we are Yien to monitor 
changes over time within the student populations. 

Evaluation of Academic Performance 

Results in the summative examinations have been used as the measure of academic 
performance. Table III details the content and format of each examination. Marks obtained 
during the preclinical years have allowed us to establish whether the academic performance 
of students electing to take part in the PBCSC differed from that of students with a 
preference for the LBC. 

The 4th year examination in 1987 and 1988 assessed the clinical sciences course, largely 
demanding factual recall and rarely assessing ability to apply knowledge. The OSCE in 
1989 and 1990 corrected this deficiency to a significant extent but expanded the content 
assessed to include the clinical attachments. As all students attended the same clinical 
attachments during the year and as clinical skills and clinical sciences learning are 
interdependent, this is not seen to be a problem. The pathology examinations were almost 
exclusively knowledge based. 

Statistical comparisons of the PBCSC and LBC groups involved the use of a non- 
parametric analysis of variance initially, with subsequent pair-wise comparisons where 
statistically significant overall differences were found. 
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Table III. Evaluation of academic performance: content and format of summative 
examinations 



2nd year 1986-89: 

* anatomy-2W/Ps*; Practical; Viva; I/C Ass * 

* biochemistry-2W/Ps; 2 Essays. 

* physiology-2W/Ps; Viva; I/C Ass. 



3rd year 1986-89: 

* preclinical neurology- W/P; Practical. 

* abnormal structure & function-W/P; Practical. 

* behavioural sciences-W/P; Assignment. 



4th year 1987-90 pathology: W/P; Essay; Practical. 

4th year 1987-88 clinical sciences: 
1 2 short answer questions, predominantly: 
1987: 9 factual recall, 3 problem solving; 
1988: 10 factual recall. 2 problem solving. 

4th year 1989-90 clinical sciences & clinical attachments: 
OSCE - 20 station 1989/ 16 station 1990, predominantly: 
1989: 2 clinical, 1 1 problem solving, 7 factual recall; 
1990: 7 clinical, 7 problem solving, 2 factual recall. 

* w/P = written paper. I/C Ass. = in-course assessment 



Outcome of the Evaluation 
Pre PBCSC Academic Performance 

The marks obtained by each student in the 2nd and 3rd year examinations were totalled, 
and the average mark for the PBCSC and LBC students were compared. No difference was 
found between the PBCSC students and those attending the LBC for any of the four years 
of the pilot programme. 

The 1987/88 Pilot PBCSC 

Given the small number of students taking part in the PBCSC and the identical format of 
the 4th year programmes for 1987 and 1988, the two classes were analysed together. 
Complete data sets were available for 107 of the 1 10 students, 27 of whom volunteered to 
take part in the pilot PBCSC. 
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The modified ADLIMS was administered at the start of the 4th year. No significant 
difference between the two groups was detected for any of the three subscales. This was 
also the case for the marks in the end of year clinical sciences examination and the final 
mark for the 4th year which includes the results of the separate pathology examination. 

Conclusions 

1 . Students opting for the PBCSC did not self-select for the PBCSC on the basis of superior 
past examination performance or due to their different approach to learning, in particular, 
a preference for the "deep approach". 

2. Students opting to attend both the PBCSC and the LBC performed as well in the end of 
year examinations as those only attending the LBC. This was so despite the PBCSC 
students having an increased workload and sitting examinations designed to assess factual 
recall rather than the application of knowledge. 



The 1989 Parallel Pilot PBCSC Programme 

Thirteen students volunteered to join the PBCSC with 46 attending the LBC. Complete 
data sets were available for 12/13 and 43/46 students respectively, an overall return rate 
of 93% for the modified ADLIMS administered at the start of 4th year. Unfortunately, the 
return rate for the ADLIMS administered at the end of 4th year was oniy just above 50% 
precluding these data from the evaluation. 

The PBCSC students scored more highly for the "deep learning" subscales 3 at the start 
of 4th year than did the LBC students. This reached statistical significance (p=0.012). No 
difference was present for subscales 1 and 2. 

Conclusions 

1 .Self-selection by students volunteering for this problem-based course has been 
demonstrated in terms of their approach to learning, but their performance in past 
examinations had not been superior. This may well reflect a far greater commitment to PBL 
and adoption of the learning skills it promotes. This is felt likely, given the risks these 
students were accepting by taking part in an untested clinical sciences course which 
contributed significantly to the final mark for 4th Year. 

2.Superior performance in the OSCE by the PBCSC students appears to support the 
hypothesis that they are better able to apply their knowledge of the clinical sciences and 
have better developed clinical reasoning skills than the LBC students. This is in part at least 
likely due to the self-selection tnat had taken place and also the effects known to occur in 
the first cohorts of students participating in innovative programmes. 
In contrast to this, the PBCSC and LBC students performed equally well in the pathology 
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examination. This supports the finding in 1987/88 that problem-based learning does not 
compromise performance in knowledge based examinations. 

The 1990 Parallel Pilot PBCSC 

39 students, 68% of the class, expressed a preference for the PBCSC at the start of their 
4th year.Due to limited resources, a ballot was held to select 18 students. Those 21 students 
not balloted joined the 1 8 students who had opted for the LBC. These events provided us 
with a quasi experimental situation and the opportunity to take PBL preference into 
account in oui evaluation of the 1990 programme. For this reason the analysis compared 
three groups; Group 1 (n=18) comprising students who attended the PBCSC, Group 2 
(n= 1 7) comprised those interested in PBL not balloted for the PBCSC, and Group 3 (n= 1 8) 
comprised those expressing preference for the LBC. 

No significant difference was found between the three groups for any of the three ADLIMS 
subscales at the start of the 4th year (90% return). We administered the modified ADLIMS 
again to this 4th Year class at the start of their 5th year. 93% of those who returned the 4th 
year ADLIMS completed it in 5th year. 

Results showed a significant overall difference for subscale 3 only. Subsequent pair-wise 
comparisons showed a significant difference between Group 1 and Group 3 (2=1.96, 
p<.05). This was also so for Group 1 and Group 2 (2=2.34, p<.02). The difference between 
Group 2 and Group 3 was not significant. 

No significant differences were found between the performance of the groups in the end 
of year OSCE or in the pathology examination. Retrospectively, the items in the OSCE 
were classified as predominantly testing knowledge, problem solving skills or clinical 
skills. The average mark obtained for the problem solving items did not differ significantly 
between the groups. 

Conclusions 

1. Those students expressing a preference for PBL and able to participate in the PBCSC 
appear to have their "deep" approach to learning reinforced by the experience. This 
contrasts with those students whose preference for PBL could not be met and who had to 
attend a didactic lecture based course. They scored significantly lower than the PBCSC 
students on the "deep approach" subscale at the end of the year. 

2. These results raise the possibility that a traditional teacher-centred didactic course has 
inhibited students from adopting the more favourable "deep" approach to learning. This 
possibility has also been raised by Coles (1985). 

3. The comparison of examination results confirm earlier reports that participation in a 
problem-based learning programme does not harm performance in knowledge based 
examinations (Schn idt et al., 1987). We were unable to demonstrate superior problem 
solving skills in those students attending the PBCSC. 
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Discussion 

Coles and Grant (1985) describe evaluation as "a deliberate act of enquiry which sets out 
with the intention of allowing people concerned with an educational event to make 
rigorous, informed judgements and decisions about it, so that appropriate development 
may be facilitated". We wonder how often this intention, which was indeed our own, is 
actually realised. 

The result of the present evaluation provide support for the favourable impact of PBL 
curricula on students' approach to learning (Coles, 198S; Newble et al., 1986). The major 
difference with this evaluation is that the effect has been demonstrated in a group of 
students attending a discreet problem-based course in an otherwise identical teaching 
programme. Importantly, the group showed no difference in their approach to learning 
from the non-PBL groups at the start of the PBCSC. The results also support the apparent 
negative effect of a teacher-centred didactic programme, previously referred to by Coles 
(1985). 

Apart from the 1989 cohort, no differences in examination performance have been 
demonstrated between the LBC and the PBCSC students. As yet we have been unable, due 
to limited resources and time, to compare the subsequent academic performance of the 
1989 and 1990 groups at the end of their 5th Year, when they sit a common summative 
examination. We intend to do this and to repeat administration of the modified ADLIMS. 

Relationship of Evaluation to Change 

We have been encouraged by these results but acknowledge that they have played no part 
in achieving the goal of replacing the 4th Year LBC with a PBCSC. Each major step in the 
process of change long preceded completion of the analysis of the evaluation. Even now, 
many Faculty colleagues will net be aware of the results. 

Certainly, in the short term, this evaluation has not fulfilled the intention outlined by Coles 
and Grant. Instead the decisions leading to change were strongly influenced by the 
increasing support for PBL by the 3rd Year students destined for Wellington. We have no 
data to suggest that the PBCSC option influenced their choice of clinical school except for 
half of the 1989 PBCSC volunteers. However, without substantial support from the 
Faculty, the decision to move to a whole class PBCSC would not have been taken. What 
were the factors influencing Faculty members, most of whom had no knowledge or 
experience of student centred PBL? 

From the start, the Dean and the Curriculum Committee supported the gradual approach 
to the development of the PBCSC. Because so few of the Faculty staff had any knowledge 
of student-centred PBL, staff development has been emphasized throughout. A system of 
"apprentice" tutors has probably been the most effective influence. Faculty involvement 
as resource persons and in the development of the PP's and later in the self-assessment 
modules has also been important. A staff questionnaire was administered at the start of the 
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1991 whole class PBCSC to determine the extent of staff support We also determined (in 
retrospect) the extent of support for the PBCSC when the decision was taken to replace the 
LBC eight months before. Support increased from 65% to 77% over the eight months. A 
two day workshop in December 1990, led by a tutor from the Newcastle Faculty of 
Medicine, Australia, contributed to this increased support in the Faculty. 

The Future 

To achieve the original goal of the PBCSC, the programme has to be fully integrated with 
the clinical attachments. Attempts to do this in 199 1 have not succeeded, due to time tabling 
issues which could not be resolved in the time available. There is general acceptance by 
the Faculty of the principle of integration which involves extending the PBCSC into the 
5th Year programme. 

Plans to evaluate the 1992 PBCSC are presently under way. Because the three Schools of 
Clinical Medicine in Wellington, Christchurch and Dunedin have adopted different 
approaches to clinical sciences teaching, we are exploring the feasibility of evaluating the 
three programmes using a similar approach to that described above but including if 
possible an evaluation of clinical competence. Potential confounding factors will be taken 
into account as far as possible. Of particular interest is the opportunity this would provide 
to study this cohort of students during the rest of their undergraduate and early postgraduate 
careers. 
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Sherbrooke Curriculum 
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Abstract 

Sherbrooke had to face another educational challenge when its first problem-based 
learning (PBL) cohort came to the clerkship, the clinical years of the undergraduate 
curriculum The Faculty took advantage of educationally well-trained academics and a 
few clinical teachers ready to innovate. Lead by respected clinicians, who became internal 
change agents, a new clerkship was designed to carry on the PBL philosophy into the 
clinical years. Bi-weekly sessions on learning clinical reasoning (LCR) in each rotations 
has become the instructional format for extending the PBL philosophy and for partially 
correcting many problems of traditional clerkship education. With the introduction of 
protected time for study, objective-structured clinical examination and definition of 
learning objectives in terms of clinical problems and pathological conditions, a better 
balance has been established between the instructional needs and the service needs, both 
components of the students ' educational experience. After a period of eighteen months, the 
new clerkship with its special educational conditions is well established. 

Introduction 

Since McMaster University introduced Problem-Based Learning (PBL) (Spaulding, 
1969) only a few new schools have adopted the PBL method. However, the GPEP Report 
(Physicians for the Twenty-First Century, 1984) has wrought an impetus of change in 
medical education, and some schools*have choosen the PBL instructional format as a 
parallel curriculum (Kantrowitz et al., 1987). Sherbrooke was the first to report a complete 
shift from its traditional course to a full scale PBL preclinical curriculum (Des Marchais 
et al., 1990; Des Marchais, 1991; Des Marchais et al., in press). 
While few schools have overtly extended PBL into the clinical years (Schwartz et al., 
1991), the PBL curriculum change at Sherbrooke introduced such a shaking-up of the 
educational milieu (Des Marchais et al., 1990; Des Marchais, 1991; Des Marchais et al., 
in press) that the clerkship of the first cohort of the new programme was seen as a golden 
opportunity to review in depth the clinical phase of its four year programme. The 1986 
planning document on curriculum reform (Le programme des 6tudes mddicales 

Requests for reprints should be sent to Jacques E. Des Marchais, University of Sherbrooke, Faculty of 
Medicine, Sherbrooke, Qu6bec J1H 5N4, Canada. 
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pr&loctorales, 1987) stated that the clerkship should be extended from twelve to eighteen 
months. In a research project in 1988 the hypothesis was raised whether it would be 
possible to continue the PBL philosophy into the clerkship, to continue to enhance student 
autonomy, and to extend the student-centred educational continuum into the clinical years. 

This paper reports on the introduction of a new instruc tional format in the clerkship, the 
sessions on learning clinical reasoning (LCR). A discussion of problems of clinical 
education and mastery of clinical reasoning, subsequent to problem-based learning in the 
preclinical period, is followed by a description of the organisational, human and material 
changes. The paper concludes with an explanation of the process of successful change and 
what might be identified as a first impact of this new experience. 

The Challenge 

The study of tlie basic sciences through a PBL preclinical curriculum is so great a change 
from the traditional approach, that it is difficult to accept that students should continue their 
studies in a traditional clerkship. Students are bound to have experienced a quite different 
preparation for the clerkship if they come from a PBL curriculum. Very little has been 
written about this expectation. Students would be expected to be more active in discussing 
clinical assumptions, to raise more questions and to suggest early exploratory hypotheses. 
They would be expected to continue to build up their bank of basic data in working through 
problems, now real clinical problems. The preclinical years should have developed self- 
directed learning which should be reinforced throughout the clerkship. The opportunity, 
indeed the need for a second educational reform, presented itself: is it possible to offer a 
student-centred clerkship by carrying on with the PBL approach? 
Numerous major problems had been identified in traditional clinical education (Physicians 
for the Twenty-First Century, 1984; McLoid et al, 1985; Morgan, 1986; Irby, 1986; 
Ambrey, 1985; Mellinkoff, 1987). Despite learning being based on clinical problems, 
students tend to be passive observers most of the time within the hierarchy of the health 
care system. Their clinical reasoning is seldom challenged, revealed or assessed in the day 
to day care of patients. The clinical work-up is mostly planned by the consultants or 
registrars in the outpatient clinic or emergency room. Observable and measurable learning 
objectives, described in specific realistic terms, are rarely available to the students. 
Clinical cases, the clinical student 9 s learning material, are somewhat unpredictable and not 
always appropriate for the current set of learning objectives. The majority of the clinical 
teaching occurs on the ward and frequently in tertiary care hospitals, when patients are 
acutely ill, present multiple, concomitant pathological conditions, and require subspecialty 
expertise and technology. This setting is disease-orientated and focuses on selective 
problems. Junior clinical students are not well prepared for patients with several diseases 
and with limited cooperation. At least in the Canadian system, other patients are admitted 
only for short stay for diagnostic or therapeutic procedures. In this setting, some major 
steps of the clinical reasoning process, that is hypothesis generation and the inquiry 
strategy may be bypassed somewhat. 
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The role of the clerk is ill defined. The balance between his traditional service tasks and 
his mandatory participation in educational activities is precarious and fragile. Not 
infrequently, his responsibilities may even be shifted away from patient care to routine 
paper work and the collection of missing reports. 

PBL is based on the early generation of hypotheses (Barrows et al., 1980; Schmidt, 1983) 
according to the hypothetico-deductive model (Elstein et al., 1978), and the inquiring 
process follows the generation of hypotheses. The PBL model differs from that used in the 
traditional clerkship where the clinical student is asked for a formal and comprehensive 
assessment of the patient's problems; traditionally, all clinical cues must first be assembled 
before any generation of hypotheses is permitted. Yet in clinical reality, the clinician 
generates early on a series of hypotheses (Barrows et al., 1980; Elstein et al„ 1978) which 
guide his search for relevant cues. A congruent, valid learning format should offer the same 
process in the clerkship. 

Although generally accepted, the hypothetico-deductive model has been challenged as a 
complete explanation of clinical reasoning. The inconsistency of performance by students 
in relation to different problems has raised the possibility that problem solving might be 
even more dependent on case-related knowledge ("case specificity") than upon the 
efficient and exclusive use of the hypothesis generation and testing method (McGuire, 
1985; Groen et al., 1985). Background knowledge is thus essential to support clinical 
reasoning. The study of specific priority problems would, therefore, seem desirable; but 
which problems within a discipline should a clerk be able to solve and which pathological 
conditions should a clerk be able to treat in order to master sufficiently that discipline? The 
definition of learning objectives in terms of priority problems and conditions is thus a 
mandatory requisite for each clerkship rotation. 

Learning is difficult if the educational format does not include feedback (Groen et al., 
1985). If the goal is mastery of the steps of clinical reasoning, the student needs to become 
competent in the intermediate or component steps, and early feedback becomes essential. 
Successful feedback, in turn, calls for thinking aloud (Elstein et al., 1978) on the part of 
the student. Verbalising the thinking process as it occurs provides the tutor or supervisor 
with reliable information without significant interference with the student's process of 
reasoning. 

In the Sherbrooke curriculum, the PBL preclinical two and a half years facilitate the study 
of the basis of medicine by integration of basic and clinical sciences through the analysis 
of problems, identification of what needs to be learned and application of such learning to 
the problem. Understanding of the underlying principles and mechanisms at the levels of 
analysis, synthesis and evaluation becomes the main focus of the instructional method. 
Clinical diagnosis is not essential at this stage and not sollicitated by PBL tutors. In the first 
semester of the third year, the focus is progressively shifted more toward problem solving, 
investigation and therapeutics. The clerkship must then offer the opportunity to progress 
in that direction and to develop mastery of the clinical reasoning process. 
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At the bedside, the iterative process of clinical reasoning proceeds rapidly. Hypotheses 
generation and testing are now assessed through a problem-orientated inquiry strategy 
using every clinical skill of information gathering, physical examination and laboratoiy 
investigation. To arrive at the right diagnosis and the best management is the raison d'etre 
of problem solving during the clerkship. To be congruent with clinical reasoning, the 
instructional format should stimulate the real clinical encounter and permit the practice of 
a rapid, iterative process. PBL, as it is used in the preclinical curriculum, had to be adapted. 

In brief, the goals of the clerkship reform are: 1 ) to establish a balance between service and 
educational needs by setting aside sufficient protected time for educational activities; 2) 
to define specific learning objectives in terms of clinical problems and pathological 
conditions relevant to each of the disciplines; 3) to make provision for variability and 
unpredictability of clinical exposure vis-i-vis the determined objectives; and 4) to offer 
educational experiences which take into account that the students have had prior experience 
of PBL and active participation, and that the students are responsible and autonomous 
learners, capable of organising their learning through personal objectives. 

Components of the Reform 

The reform of the clerkship has been aimed towards an increased exposure to primary care 
and community-based hospitals; a better definition of specific objectives in terms of 
priority problems and conditions; the development and introduction of a new educational 
format; bi-weekly sessions on learning clinical reasoning (LCR); the implementation of 
protected time which ideally represents about 15 to 20% of free daily time for personal 
work and integrated study pertinent to the patient which the student has seen; and 
development of congruent student assessment. A brief description of the composition and 
sequence of the clerkship rotations will be followed by a discussion of the objectives and 
the sessions on LCR. The Sherbrooke clerkship lasts sixty eight weeks (Table I). 

After three months of rotations in community health and two electives, the regular 
clerkship disciplines are spread over one year, so that two cohorts of students do not 
overlap. In the regular rotations, seven weeks of family medicine and walk-in clinic care, 
and a new three and a half week of a multidisciplinary rotation (including ENT, 
ophtalmology and dermatology) have been introduced. The remaining period is used for 
rotation through the traditional disciplines (medicine, surgery, obstetrics and gynaecology, 
paediatrics, and psychiatry) with the inclusion of the new educational activity - the sessions 
on learning clinical reasoning (LCR). 

Learning Objectives: Priority Problems and Conditions 

Eighteen months before the start of the new clerkship, the mandate of the educational 
reform was accepted by an ad hoc committee consisting of the clerkship coordinator, 
influential academics and clinical teachers with specific background in medical education. 
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Table I. Clerkship rotations 



January - April 

Community Health 
Elective 1 
Elective 2 



4 weeks 
4 weeks 
4 weeks 



April to March 

General Internal Medicine 
* Multidisciplinary 
Family Medicine 
and Walk-in Clinics 



3,5 weeks 
3,5 weeks 

7 weeks 



Vacation - July 



1 week 



Surgery 
Paediatrics 

Gynaecology-Obstetrics 
Medical subspecialty 



7 weeks 
7 weeks 
7 weeks 
7 weeks 



Vacation - Christmas 



2 weeks 



Psychiatry 



7 weeks 



April 1 to May 12 

Preparation for National Certification Examination 



May 13 - June 7 
Elective 



3 weeks 



TOTAL 



68 weeks 



Multidisciplinary: ENT, ophtalmology, dermatology. 
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They were asked to develop appropriate educational experiences. Their first task was to 
identify mandatory learning objectives. 

It was agreed from the start that each rotation should define specific objectives in terms of 
priority clinical problems and pathological conditions. Lists, such as that of the Medical 
Council of Canada (Medical Council of Canada, 1985), were used to compile a set of 
problems and conditions by each department and these were validated by the special ad hoc 
committee. 

The mandatory clinical problems were then linked with the LCR sessions which cover all 
the priority problems. In addition to these problems, a list of pathological conditions had 
to be specified (Table II). 

Table II. Priority problems and conditions 



Problems Pathological Total 
Condition 

Surgery 14 45 59 

Medicine 19 42 61 

Gynaecology /Obstetrics 13 47 60 

Paediatrics 14 43 57 

Psychiatry 7 21 28 

Total 67 198 265 



A proportion of these conditions is also explored through the LCR sessions. The 
pathological conditions that are not discussed during the LCR sessions are expected to be 
studied in the context of the students' regular clinical activities and individual work during 
the rotation. At the end of the five regular rotations, the students have to demonstrate their 
ability to generate and test diagnostic hypotheses, to plan an appropriate laboratory 
investigation and to delineate the principles of therapy for each of the 67 clinical problems 
and the 198 pathological conditions (Table III, Table IV) 
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Table HI. Example: Priority clinical problems for rotation in medicine 



1 

1. 


v^nesi pain 


1 1 

ii. 


C 1 ATI IjftfYC 

owoiicn ICgd 


2. 


Dyspnoea 


12. 


Black stools 


3. 


Cough 


13. 


Hypertension 


4. 


Syncope 


14. 


Diarrhoea 


5. 


Haemoptysis 


15. 


Jaundice 


6. 


Shock 


16. 


Fever and chills 


7. 


Headache 


17. 


Acute renal failure 


8. 


Fatigue 


18. 


Weight loss 


9. 


Coma 


19. 


Arthritis 



10. Confusion 



Table IV. Example: Twenty two of fourty three priority pathological conditions for the 
paediatric rotation 



1. 


Hyaline membrane disease 


12. 


Immunisation 


2. 


Jaundice in the newborn 


13. 


Battered child syndrome 


3. 


Hypoglycaemia in the newborn 


14. 


Acute pharyngitis 


4. 


Cystic fibrosis of the pancreas 


15. 


Acute otitis media 


5. 


Asthma 


16. 


Sinusitis 


6. 


Cardiac failure 


17. 


Varicella 


7. 


Innocent heart murmur 


18. 


Measles 


8. 


Mental retardation 


19. 


Rubella (German measles) 


9. 


Cerebral palsy 


20. 


Roseola infantum 


10. 


Febrile convulsion 


21. 


Scarlet fever 


11. 


Colic in the infant 


22. 


Whooping cough 



Assessment 

We also have modified the assessment system to be congruent with the objectives. At the 
end of every rotation, an objective-structured clinical examination (OSCE) (Harden et al., 
1979; Van der Vleuten et al., 1990) has been developed that focuses on the assessment of 
different steps in the clinical reasoning process. Three major types of stations are included 
in the OSCE for the priority problems and condition studied during the rotation. In the first 
two types of OSCE stations the student is faced with a simulated patient presenting a 
clinical problem for which the student has to undertake a focused history or physical 
examination and demonstrate hypothesis generation and a hypothesis-driven inquiry 
strategy. The other type of station is a structured oral examination based on a clinical 
vignette where the student has to elaborate a pertinent differential diagnosis, as well as 
appropriate laboratory investigations and therapy. 
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Learning Clinical Reasoning (LCR) Sessions 

It became evident to the ad hoc planning committee that a sy stematic* standardised learning 
activity had to be developed, if the curriculum was to carry the PBL approach into the 
clerkship. During the first two years of the programme at Sherbrooke the PBL learning 
activity (Des Marchais et al., 1990; Schmidt, 1983) is focused only on analysis and an 
understanding of problems and does not include discussion of diagnosis and therapy. Such 
discussions do not begin until the first semester of the third year. We accept that the PBL 
method is primarily intended for the study of the foundation of medicine by integrating 
basic sciences with an understanding of clinical problems, arranged in the organ system 
curriculum design. In the clerkship, students undertake the other phase of the study of 
medicine, the long process of mastering clinical reasoning. 

We, therefore, designed the new clerkship instructional format, the sessions on learning 
clinical reasoning (LCR). The educational intention is to give students the opportunity to 
practise clinical reasoning on specific problems. The students are encouraged to practise 
the intellectual activity of thinking aloud. This requires them to be active, to receive 
continuous feedback and to shape their clinical reasoning. These activities favour a more 
conscious process of thinking and enhance the students' autonomy. 

The session of LCR lasts an hour and a iialf on a bi-weekly basis. The goal is for the students 
to learn the process of working up simulated clinical cases, in order to master investigation 
and treatment of patient problems. This learning format was also developed to counteract 
unpredictable clinical exposure and the great variability of clinical opportunities, ard to 
give each student a fair, standardised and systematic opportunity to confront each of the 
67 priority clinical problems. It became necessary to develop a bank of these clinical 
problems and pathological conditions. For each of the priority problems, a clinical vignette 
was written with all the pertinent positive findings of history, physical examination and 
laboratory investigation. These data are documented together with occasional clinical 
records, such as radiographs. Each case has to be sufficiently complex and rich to satisfy 
the educational intentions and the goal of learning clinical reasoning. However, the 
documentation has to be sufficiently succint to trigger and complete discussion in the 
available time. All cases concentrate on priority problems and conditions and they are 
based on real patients to provide realism and to facilitate the use of radiographs and 
laboratory data. 

During a rotation, a group of five to eight clerks meet with a clinician who acts as the 
resource expert in the discipline of the rotation. One clerk takes charge of the "data bank"; 
he will have familiarised himself with the clinical case in advance. He will give specific 
information as his fellow students request it One student takes on the role of the 
interviewer. He begins to generate hypotheses and tries to obtain pertinent information in 
an iterative approach (Kassirer, 1983). The student with the "data bank" will reveal 
information only as it is specifically requested, and each question has to be justified by the 
thinking-aloud-procedure which is monitored by the clinical instructor. As the process 
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moves on towards physical examination and investigation procedures, the hypotheses are 
reviewed and reexamined with new data. The instructor observes the process and shapes 
the clinical reasoning steps as they relate to his own specialty. Once, the interviewer has 
completed the exploration of his hypotheses, he hands his role on to another student who 
will, in turn, pass the task on to another student, and so on. In this manner, everybody 
participates in the session. While the individual reasoning process is respected, other 
students may be invited to participate or they may do so at their own request. In this manner 
the problem is analysed into a short list of pertinent hypotheses (differential diagnosis). 
Subsequent laboratory investigations are selected to eliminate or prove the potential 
diagnosis. When the probable or proven diagnosis has been selected, the treatment is 
planned accordingly . Throughout the progress the instructor corrects, initiates and enhances 
particular steps in the clinical reasoning process . The group, lead by the instructor, will then 
proceed to generalise and to develop a systematic and logical approach to the priority 
problem and the condition that have been discussed. This step is critical to facilitate 
transference of knowledge and process to further and different clinical cases. At the end 
of the session, the sroup will spend a few minutes to review the process of the group's 
learning. Finally, each student is asked to define his personal learning objectives for which 
he will have the responsibility to achieve mastery through his own further studies. 

As during the preclinical PBL, one student will assume the role of secretary and write on 
the blackboard the important elements at each step of the process. The clinical instructor 
plays two main roles. He leads the group and stimulates student participation to sustain the 
process in an enjoyable climate. Secondly, the instructor is the role model for clinical 
reasoning which may differ with different disciplines. He initiates, assists, stimulates, and 
reorientates the student who is engaged in hypothesis generation, information gathering, 
planning investigation or therapy. 

There are clear differences between the preclinical PBL and the clinical LCR sessions. In 
the preclinical years the students at Sherbrooke learn by being confronted with contextual 
clinical problems that have been arranged by organs and systems. The students structure 
their knowledge around problems and they are encouraged to use the basic sciences to 
reach optimal explanations for the problem. During the clerkship, students continue to built 
their knowledge around problems, this time, however, divided into clinical disciplines. 
Here, they apply their knowledge more specifically to the clinical reasoning steps towards 
the best solution (not only explanation) of the problems. Here the clinical reasoning 
sessions place emphasis first on application of previous learning and only secondly on 
identification of the need for further learning. 

Implementation and First Experience 

The decision to continue the PBL approach in the clerkship was taken by the Vice-Dean 
for Education, the Director of the M.D. programme and the Clerkship Coordinator. The 
planning of the new undergraduate PBL curriculum in 1987 included a modification of the 
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clerkship (Des Marchais, 1991; Universit6de Sherbrooke, 1987). It was, therefore, logical 
to complete the mandate. The internal change agents had the clear perception that they 
should not miss the golden opportunity of the first cohort coming up to their clerkship and 
the generally favourable climate for educational innovation. Moreover, the clerkship 
coordinator was in the process of acquiring educational expertise and he was helped in his 
task of reforming the clerkship by a young clinician who had joined the Faculty with a 
brand new Masters degree in Education. These colleagues worked as a team. The proposal 
for reforming the clerkship was then accepted by all the administrative committees. 

The institution had invested a great deal in the educational development of its academic 
staff (Grand'Maison et al., 1991). Clinical academics who were already involved as PBL 
tutors perceived the new clerkship as the natural evolution of a student-centred curriculum 
and became early participants in this endeavour. Each major clinical department had at 
least one clinician who had acquired some educational expertise through the one year 
training programme (Des Marchais et al., 1 990) and who was already involved in education 
for a fair proportion of his professional time. The clerkship reform was, therefore, based 
from the beginning on a critical mass of competent and collaborative colleagues. It was 
thus not too difficult to develop the bi-weekly sessions on LCR. Each department 
rominated one clinician to be responsible for the construct of the case bank, for planning 
the implementation and for helping his peers to become LCR instructors. As the LCR 
instructional method is so close to normal clinician practice, it was easy to replicate the 
same thinking with students in the sessions on LCR. To spread this innovation, members 
of a department were invited to a special meeting where the new format was explained and 
illustrated with a videotape. 

Just as the implementation of the preclinical PBL was accomplished without using a 
parallel experimental track (Des Marchais, 1991), a complete revision of the clinical 
course was choosen as the model for change. From the very first day, each department had 
to offer bi-weekly sessions on LCR. The strength of the demand for this change and student 
support, together with some improvisation made it work successfully. Eighteen months 
later, the LCR sessions have become an integral part of the rotations in all five major 
disciplines. Some clinical problems that have not been analysed in any depth, were now 
systematically thought through. 

Formal student evaluation questionnaires are administered after each rotation. The 
response rate is around 90%. First analysis of the results revealed that LCR sessions are 
greatly appreciated by the students who attend virtually all sessions. The LCR sessions 
have partially replaced other educational activities of the former clerkship. After the initial 
investment of time and resources for the development of the bank of clinical cases, the 
OSCE and the training of colleagues, the new clerkship is now supported at no additional 
cost. The perception of the instructors is generally positive. 

After one year of sessions on LCR, the major difficulty is to standardise the method 
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throughout the five different disciplines. The numerous different instructors and the 
different teaching hospitals, where these sessions take place, may in part explain the 
heterogeneous application of the LCR methodology. Occasionally a session may deviate 
and become a mini-lecture or assume a seminar format. The principles of the LCR session 
must remain but their application may differ according to the nature of the discipline itself. 
It may even differ according to the instructor. Yet to be exposed to the individual clinical 
reasoning process of numerous experienced clinicians is a powerful way of learning. Some 
minor variations seem to be useful, in order to meet specific requirements of a particular 
discipline. As an example, in psychiatry the instructor plays the role of the "data bank" in 
order to provide not only the basic facts of the clinical case but also the non-verbal and 
gesticular cues. The sessions are also longer in that discipline (2 hours) to permit the 
complete and realistic unfolding of the clinical situation. Another example is paediatrics 
where background knowledge may be insufficient for some problems (e.g. battered child). 
Here the students might have to do some preparatory readings to develop sufficient 
structured knowledge to support the process of clinical reasoning. 
Scheduling was also an initial problem for the medicine and surgery rotations where the 
clerks who are engaged in different subspecialties have to be brought together in order to 
participate in the same LCR sessions. 

At present the educational activities are well implemented. Additional effort will be 
devoted in the near future to the standardisation of the method, the updating of the clinical 
problems and do the refinement of the instructor's role. Similarly, the OSCE will be 
enriched with new stations and with the regular use of standardised patients. 
By and large, this second major curriculum change has not been too costly in terms of the 
teachers' time. The sessions on LCR have replaced the numerous lectures of the traditional 
clerkship. The implementation of OSCE was a necessary complement, and the clinicians 
like to check real performance, even though more time is involved. We took advantage of 
the momentum engendered by the preclinical reform, and many young clinicians used the 
clerkship reform to gain points toward their academic promotion. In brief, the staff is 
generally satisfied with the new clerkship where the students are now much better able to 
express their clinical reasoning. 
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Abstract 

This paper describes and evaluates the role of a non-governmental organisation, the 
King p s Fund Centre, in bringing about change in undergraduate medical education in the 
UK. The chosen strategy, a national enquiry using the Delphi technique, is outlined in 
terms of the method and process. The outcome is described, and the success of the King 's 
Fund Centre as a change agent is analysed 

Introduction 

The King's Fund Centre (KFC) is a health services development agency which promotes 
improvements in health and social care with the aim of ensuring that good developments 
are widely taken up. The Centre is part of the King' s Fund, which is an independent charity 
which seeks to stimu ate good practice and innovation in all aspects of health care and 
management through research and development, education, policy analysis and direct 
grants. One of the main areas of interest of KFC is the future provision of services in 
London, including the ftiture delivery of acute services and the interface between hospital 
and community. Recent developments in medical care and the re-organisation of the UK 
National Health Service, on the basis of managed competition, have demonstrated the need 
for a new programme of development work focusing on the interface between health 
service delivery and undergraduate medical education in relation to clinical teaching. 

Many medical schools have been experiencing problems in carrying out clinical teaching 
that is traditionally centred around bedside teaching on the wards. These problems are due 
to changes in medical practice, for example new techniques which allow patients to be 
diagnosed and managed as outpatients, and in health service delivery, for example shorter 
stay in hospital . Some of these changes conflict with the requirements for teaching students 
who need time to talk to, and examine patients. The conflict between patient care and 
teaching has been particularly noticeable in London. This is largely related to the 
oversupply of large teaching hospitals in tie centre of the city in contrast to the diminishing 
population in inner London while about a third of the country's medical students are 
educated in London. 
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In 1989 the King's Fund Centre became interested in medical education. It had been 
suggested that the medical schools were acting as a brake on the development of more 
appropriate health services for London. An additional reason was anxiety over the likely 
impact of the government* s health service reforms that were due to be introduced early in 
1 990. The Centre, therefore, responded favourably to a combined request from the City and 
Hackney Health Authority (one of the London District Health Authorities) and St. 
Bartholomew's Hospital Medical College (SBHMC), one of the London medical schools, 
to jointly fund a project to develop new methods of clinical teaching that would be more 
in line with future health service delivery. 

The two-year project began in January 1990 with a review of the problems in clinical 
teaching. It soon became apparent that, before new ways of clinical teaching could be 
developed, it would be necessary to define more clearly the aims of clinical teaching for 
undergraduates. It was, therefore, decided to mount a national enquiry into the future of 
undergraduate clinical teaching toestablish guidelines for the design of future undergraduate 
curricula in which clinical teaching would be adapted to the changing needs and 
circumstances of health care. By using the changes in the health service as a lever, this was 
seen as an opportunity to widen the scope of the project to facilitate a process of change 
in the whole of undergraduate medical education. The enquiry thus became part of a wider 
strategy to create a favourable climate of opinion and momentum for change. This paper 
describes the enquiry, its findings and outcome. It also analyses the success of the strategy 
in promoting change in medical education in the UK. 

Method of the Enquiry 

Agreement to proceed with the enquiry as a King's Fund Centre initiative in collaboration 
with St. Bartholomew's Hospital Medical College was reached early in 1990. As the aims 
were to develop a consensus view of the future of undergraduate clinical teaching and to 
create a favourable climate of opinion for such change, it was decided to use a modification 
of the Delphi technique, first developed in 1944 at the Rand Corporation (Delbecq et al., 
1975). The Delphi Conference is a method of obtaining the collective view of a group of 
experts through repeated rounds of responses to a questionnaire with anonymous feedback 
of the responses of colleagues from previous rounds. It enables the development of a 
consensus view about a complex issue and ensures a certain feeling of ownership of the 
final product. 

In Round I participants were sent a consultation document containing a series of propositions 
and questions on various aspects of the planning, conduct and evaluation of undergraduate 
teaching. In this Round they were asked to add any other issues which they thought that 
the enquiry should address. They were asked not to give any comments at this stage. As 
a result of the responses, the consultation document was amended, and additional 
propositions and questions were included. 
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In Round II the participants were invited to comment on the enlarged set of issues by 
agreeing or disagreeing with the propositions (with their reasons) and by giving their 
answers to the questions. The comments and answers were summarised in 

Round m, so that the participants could see the range and degree of consensus of views 
which had been expressed. They were also invited to make additional comments or new 
suggestions in the context of the collated responses from Round n. 
The propositions and questions for the first round of the Delphi conference were developed 
by a small group comprising people with expertise in medical education, midical 
academics from St Bartholomew's Hospital Medical College and staff from the King's 
Fund Centre who were familiar with Delphi methods. Round I was sent out in May 1990, 
with a personalised covering letter from the Director of the Centre and the Dean of the 
Medical College. 

The document was addressed to the deans and heads of major clinical departments in all 
28 British medical schools, and to senior representatives of the Royal (postgraduate) 
Colleges. A group of 1 0 students and 1 0 pre-registration house officers (recently graduated) 
from each of two medical schools (St. Bartholomew's and University of Southampton) 
were also contacted. Reminder letters were sent out some four weeks later. Of the 344 
individuals invited to participate, 213 (62%) responded to Round I. 

Round II was sent out in July, again with a personalised letter signed by the Director of the 
Centre and the Dean of the Medical College. In order to ensure a maximum response rate, 
the initial deadline of October 1 st was extended, and two reminder letters were sent out in 
mid-September and late October respectively. Responses were received from 192 (56%) 
of those invited to participate. 

Correspondence from participants showed that there was a strong interest in having an 
opportunity to discuss the issues further. This reinforced the strategic plan to follow the 
enquiry with a one day meeting to discuss the findings. 

Round III was sent out in December, along with an invitation to a one day discussion 
meeting for April in the following year (1991). Twenty-eight people provided additional 
comments at this stage, and over a hundred indicated an interest in attending the meeting. 

The responses received in Rounds II and m were summarised to form the discussion 
document for the meeting. It presented the consensus view on the aims of a possible new 
curriculum and the recommendations in relation to ten sets of issues. The conference was 
attended by 86 participants who endorsed the findings of the enquiry and considered, in 
small groups, the ten sets of recommendations. 

The strategy included the formation of a working party to maintain the momentum 
generated by the Delphi Conference and the one day meeting. Accordingly, the major areas 
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which had been identified for immediate action were discussed by a small working party 
approximately one month after the meeting. Key people from the various interest areas in 
medical education met with a view to arriving at a plan of action. In July of the same year 
a definitive report was published (Towle, 1 99 1). This consisted of a summary of the Delphi 
enquiry findings that had been produced for the one day meeting, together with summaries 
of the relevant group discussions at the meeting. In addition, the report identified the key 
issues which needed to be addressed if change in undergraduate medical education were 
to be initiated and maintained. 

Process of the Enquiry 

The process of the enquiry, outlined above, is illustrated in Appendix A and shows the 
development of one of the propositions through the different rounds of the Delphi 
consultation and to its eventual appearance in the final report. 

Round I of the enquiry consisted of 13 propositions and 6 questions arranged in two 
sections: What are the purposes of undergraduate clinical teaching ? and How can clinical 
teaching achieve its aims? The first section was further subdivided into 3 subsections: 
General considerations; Overall aims; and Specific aims. 

In Round II, as a result of suggestions in responses to Round I, the numbers of propositions 
were increased from 13 to 14 and the questions were increased from 6 to 13. Additional 
questions about consequences for curriculum planning were posed at the end of each 
subsection. 

Comments contributed in Round II were analysed by classifying the responses to the 
propositions into three groups: agree entirely with the statement, agree but with certain 
reservations and disagree. 

In Round III, the three major arguments advanced for each of the categories were 
summarised, together with the number of respondents respectively. Any remaining 
minority comments were included in a related Appendix. Answers to the questions were 
classified and listed in order of priority. While the main points were summarised in the text 
of Round III, additional points went into the Appendix. A separate section was created to 
accommodate all the responses to the questions on consequences for curriculum planning. 
These summaries were grouped under the following headings: general considerations; 
overall approach to planning; overall structure; content; process; organisational structure; 
and resources. 

The discussion document for the one day meeting reviewed the findings of the enquiry, in 
order to summarise the consensus views on the aims of a possible new undergraduate 
curriculum and the requirements for achieving those aims. A separate section identified 
recommendations for future action if the consensus were to be implemented. These 
recommendations were grouped into 10 themes: aims of the curriculum; structure of the 
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course; how students should learn; student assessment; student selection; the pre- 
registration period; where students should learn; quality of teaching; curriculum plan- 
ning; and organisational requirements. 

Outcome 

The Delphi enquiry and the one day meeting both indicated that large numbers of 
academics wanted to see major change in the undergraduate curriculum. The evidence also 
indicated that there existed a consensus view of what that change should consist of in terms 
of the design of a curriculum for the future. While it is not the purpose of this paper to detail 
the findings of the enquiry, which have been reported elsewhere (Towle, 1 99 1 ), the general 
outcome is presented in Table I. 

Table I. Summary of findings - principles which should inform the curriculum of the future 



. Reduction in factual information. 

. Active learning (enquiring doctors). 

. Principles of medicine (core knowledge, skills and attitudes). 

. Development of general competences (e.g. critical thinking, problem solving, 
communication, management). 

. Integration (vertical and horizontal). 

. Early clinical contact. 

. Balance between hospital/community; curative/preventive. 

. Wider aspects of health care (e.g. medico-legal/ethical issues, 
health economics, political aspects, medical audit). 

. Interprofessional collaboration. 

. Methods of learning/teaching to support aims of curriculum. 
Methods of assessment to support aims. 



There was widespread agreement that present curricula are grossly overcrowded with 
factual information which soon becomes out of date and inhibits students from developing 
into creative, critical thinkers and problem solvers who are enthusiastic about continuing 
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to learnT The comprehensive system of postgraduate education which now operates in 
Britain should free the undergraduate curriculum to concentrate on enabling students to 
develop the core knowledge, skills and attitudes which will last a professional lifetime, 
irrespective of the graduates 9 subsequent choice of specialty. 

The dissemination of the findings of the enquiry came at a time when the General Medical 
Council (GMC) sent out to all medical schools the conclusions of its own working party 
in the form of a consultation paper. This document woac' eventually become the 
Recommendations on Undergraduate Medical Education which the GMC has a statutory 
duty to issue. Although the information for the enquiry and the GMC paper was collected 
in very different ways, there was a striking concordance in the views expressed. The GMC 
paper recognized that the previous Recommendations (1980) had reflected many of the 
views expressed in 1991, but that very little change had occurred in the intervening ten 
years. The GMC were thus concerned to identify now some of the barriers to change. 

It was clear from discussions at the KFCs one day meeting and those of the working party 
that, while the enquiry had resulted in a consensus view about the changes that were 
required, there was a disappointing lack of ideas about how to proceed with implementation 
and very little understanding about the process of change. 

Therefore, in the final report of the enquiry attention was paid to the management of 
change, to identifying the forces for and against change, the key people involved and some 
of the major issues which, if tackled, would greatly facilitate change (Table II). 

One of the major problems in trying to take these issues forward was the lack of any key 
group or institution that would be in a position to take a strong lead. While deans were 
accountable to universities and to the GMC for the implementation of its recommendations, 
their collective body, the Conference of Deans was weak and unresourced. The GMC, 
while having a statutory role in undergraduate education had limited resources, expertise 
and will to push hard for reform. There was, therefore, an important continuing role for the 
KFC to play in keeping the momentum going and in supporting and strengthening those 
who could lead the way. 

In the six months since the publication of the report of the enquiry, the KFCs work, the 
GMC document and pressure from the health service and other sources have acted and 
reacted together to result in a number of developments which confirm that the momentum 
has not only been maintained but has increased and become firmly established. The deans 
have agreed to set up a Council of Deans with a secretariat which will enable them to take 
a stronger lead in curriculum matters. The GMC has begun a series of consultative visits 
to all medical schools to discuss their proposals and be informed about each school's plans 
and problems. Many articles about medical education have appeared in the medical and 
health service press, thus increasing awareness of the issues and promoting the idea that 
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change is timely and inevitable. Teaching staff in many medical schools are requesting 
assistance in educational matters, and the demand for staff development activities is 
growing. Even the seemingly intractable problem of lack of recognition for teaching is 
being addressed, with some deans including teaching requirements and standards into 
contracts for academic staff and promoting staff on the basis of excellence in teaching. 
Amid all these changes the KFC is continuing to play a facilitative role by creating a 
network for those interested in change in medical education, in order to provide a forum 
for discussion, exchange of information and ideas, and the sharing of good practice. 

Table II. Summary of findings - action required 



The following key issues must be addressed in relation to curriculum design (1-4) and to 
implementation of change (5-8), if major change is to occur in undergraduate medical 
education. 

1. Definition of the core knowledge, skills and attitudes which undergraduates need to 
acquire in relation to what a new graduate is expected to be able to do. 

2. Integration of teaching, both horizontally between clinical disciplines and vertically 
between preclinical (basic, behavioural and population sciences) and clinical sciences. 

3 . Introduction of self-directed learning, in order to encourage students to take responsibility 
for their own education as undergraduates and throughout their professional career. 

4. Development of appropriate systems of assessment to support the aims of the curriculum. 

5. Recognition for teaching, so that it is perceived as an important activity comparable in 
status to clinical, research and management activities. 

6. Training for teachers/staff development, so that curriculum development, teaching and 
assessment are carried out professionally and that all staff subscribe to the aims of the 
curriculum. 

7. Definition of where students should learn, in order to achieve the aims of the 
curriculum, and consideration of resource and logistical implications. 

8. Management of change within medical schools to facilitate the introduction and 
continued development of new curricula. 

Analysis of the Strategy 

Since the KFC began its involvement with undergraduate medical education in January 
1990, many developments have occurred, some as a direct result of the KFC's work and 
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others because of concurrent activities and pressures. It is not possible to gauge the impact 
of the KFCs strategy in isolation, but it is clear that the time was right for change and that 
the enquiry identified, encouraged and shaped the direction of change. The basic strategy 
of using necessity (the inevitable changes in the health service) as a hook on which to attach 
the enquiry, rather than starting from a philosophical base (such as 'the role of the doctor 
in the 21st century*) was clearly important. It demonstrated to those who might otherwise 
have been reluctant to acknowledge the need for curriculum reform the relevance of 
reviewing the fundamental aims of undergraduate education. At a later stage in the process, 
the consultation document from the GMC gave further confirmation that these issues were 
not going to go away. The similarity between this document and the findings of the Delphi 
enquiry increased the credibility of both and so further reinforced the direction of change. 

In addition to these 'external* factors which added to the success of the strategy for change, 
a number of 'internal' factors can be identified which contributed to the success of the 
enquiry in its own right. Firstly, there is the reputation of the King's Fund Centre as an 
independent and knowledgeable organisation which has expertise, credibility and, it must 
be said, financial and other resources. Several of the participants in the enquiry commented 
on the need for such an organisation to interest itself in medical education, as none of the 
other potential leading bodies were held in high enough esteem or were perceived to be as 
independent. The credibility of the KFC in undertaking this work outside its normal area 
of operation was further enhanced by working in close collaboration with one of the 
medical schools. A number of participants said that they took part because of their personal 
respect for the dean of that medical school. It was, therefore, an important, if small part of 
the strategy that personal letters of invitation were sent out from both the Director of the 
KFC and the Dean of SBHMC. 

This personal invitation, coming from respected persons and organisations, was necessary 
in view of the need to encourage a high rate of participation among very busy academic 
staff in what was a time-consuming and demanding process. The enquiry was initially 
viewed with scepticism by a number of those who took part somewhat reluctantly in view 
of the time needed to complete the various rounds. The turning point came when Round 
III was sent out and people could see the results of their time and effort in the 
comprehensiveness of the responses set out in a comprehendible fashion. Many commented 
at this stage that they thought Round III was a very useful document. It was evident that, 
even in its somewhat unwieldy form, it was already being used by chairmen of curriculum 
review committees and heads of departments to stimulate and focus debate. Although 
some participants feared that the enquiry would be just another paper and pencil exercise 
and were not convinced that anything would change, the determination of the KFC to 
continue to play a role in medical education in a non-threatening and facilitative way 
provided reassurance and hope. 

The major disadvantage of the strategy was that it did not involve basic scientists in the 
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process from the outset Initially the enquiry was focused solely on clinical teaching in 
keeping with the original terms of reference of the KFC's involvement Although the 
enquiry contained some propositions which referred to the overall aims of the undergraduate 
curriculum and to the integration of basic and other sciences with clinical science, those 
invited to participate were entirely from clinical departments. This proved to be a problem 
at a later stage when it became clear that the enquiry was having a wider impact than had 
been envisaged originally. Although ownership of the ideas emerging from the enquiry 
was clearly established among clinical academics, further work needed to be done to 
include basic scientists at this fairly late stage in the debate without further emphasizing 
the divide which already exists between preclinical and clinical teachers. Ways of 
addressing this difficulty have been sought through the deans, some of whom are basic 
scientists, and also through the general publicity which the enquiry has engendered, both 
nationally and within individual medical schools. 

Conclusion 

The Delphi enquiry and subsequent activities by the King's Fund Centre demonstrated the 
key role which an independent non-governmental organisation can play in creating a 
climate of opinion for change in medical education. By using changes and problems in the 
health service as the starting point for a review of the aims of the undergraduate curriculum, 
rather than a purely philosophical starting point, the need for change was made relevant, 
concrete and unavoidable. The independence, credibility and high standing of the facilitating 
organisation was another key element in the success of the strategy. The enquiry and its 
short-term outcome have provided a sound basis for further work by the King's Fund 
Centre in facilitating change in medical education, particularly in relation to the future 
provision of health services. 
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Appendix 

Example of the development of a proposition through the various stages of the enquiry. 
Round I 

(This Round asked respondents to suggest additional propositions and questions.) 
Proposition: 

As an essential major aim of this general medical education, a number of key competences 
should be developed by the time of graduation. These general competences will be 
essential for the entire period of professional life, irrespective of the specialty in which the 
graduate eventually practises. These competences will include: 

Applying the scientific method and critical thinking. 

Problem-solving, including dealing with probability and uncertainty. 
. Information gathering, manipulation and storage. 
. Managing self and with others. 

Communicating. 

Independent learning. 

Round U 

(This Round invited respondents to react Uj the propositions and to answer the questions.) 

Proposition (Revised as a result of additions suggested by respondents in Round I): 
As an essential major aim of this general medical education, a number of key competences 
should be developed by the time of graduation. These general competences will be 
essential for the entire period of professional life, irrespective of the specialty in which the 
graduate eventually practises. These competences will include: 

Basic clinical skills (interviewing, physical examination). 

Communication skills. 

Applying the scientific method and critical thinking. 
. Problem-solving, including dealing with probability, ambiguity and uncertainty. 
Information gathering, manipulation and storage. 

Management of self and others, including proper use of available resources and 
working within an organisation. 
Working with others as a team. 
Interpersonal relationships. 
Independent learning. 

Adaptation to change and participation in change. 
Several of these general competences are intimately dependent on a sound foundation of 
liberal studies. 
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Round m 

(This Round invited respondents to comment further, now that they were presented with 

a survey of responses from all respondents.) 

[Proposition as stated for Round II] 

Agree: 123 Agree - but: 57 No comment: 2 

(With representative sample responses): 

Agree 

. These are the basic competences which can serve a doctor for a professional lifetime 
and upon which specific skills and knowledge can be built during training. Such 
competences would place any medical graduate in a position to continue learning (it is 
not possible to teach a body of facts which will serve a professional lifetime), solve 
problems and contribute to individuals and society in general. 

. These competences are fundamental if future doctors are to be able to cope with a) 
change and development in clinical practice and b) multiprofessional aspects of health 
care in which the doctor will be a team leader effectively managing resources and 
making the best use of services within financial constraints. The ability to accept and 
initiate change will be crucial. 

. Besides a basis in the liberal studies, a number of these competences have to be taught 
in a structured setting. Medical education must provide appropriate opportunities for 
the achievement of these general competences by explicit consideration of their 
importance and relevance during clinical training. 

Agree - but 

The main area of contention was the final sentence ("Several of the general competences 
are intimately dependent on a sound foundation of liberal studies"). It provoked two sorts 
of comments: 

. What is meant by liberal studies? 

. The last sentence is a non-sequitur. liberal studies are not needed for development of 
these general competences which can easily be learned within the context of medical 
education. 

Some competences are more important than others: some are vital for the job (e.g. 
clinical skills, problem solving) some are not vital but will improve performance, some 
will not necessarily be practised in a future career. Competence in medicine (practical 
skills) should have priority over all other considerations. Some of these competences 
are more effectively learned whi!r practising and are therefore more appropriate for the 
postgraduate than undergraduate period. 
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Some competences are dependent on personality/character/upbringing/life experiences 
and cannot be taught. It is therefore important to select students carefully. 

Discussion Document/Final Report 

Undergraduate education should provide appropriate settings for the acquisition of a 
number of key competences which will be required throughout a professional lifetime, 
irrespective of which specialty is practised. The competences will enable future doctors to 
cope with change and developments in clinical practice (by adapting to and participating 
in change) and with their wider responsibilities (e.g. as team leader, manager of resources). 
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PROBLEM-BASED LEARNING 

Materials for Problem-Based Learning Sessions. The Faculty of 
Medicine, University of Limburg 

Hetty Snellen-Balendong 
University of Limburg 
Maastricht, The Netherlands 

Abstract 

This paper gives a short overview of the structure of the curriculum of the medical school 
in Maastricht, the Netherlands. The learning activities of the students in the first four years 
of the curriculum are especially stressed and some examples of tasks used for small group 
sessions are shown. 

Introduction 

Since its establishment in 1974 the Medical School at Maastricht has adopted a problem- 
based learning curriculum. It consists of a six year programme, including two years of 
mimical and primary care rotations. Although numerous changes have been introduced in 
the management of education, the format of the learning units, the design of the curriculum, 
the format of the tasks, etc., the basic philosophy of problem-based learning has never been 
challenged. A short overview of the current "state of the art" and some examples of the 
materials used in problem-based learning sessions are reported here. 

The General Concept 

The learning process in the first four years of the curriculum is centred around small group 
sessions. A two-hour group session is held twice a week with 8 to 10 students under the 
guidance of a Faculty tutor. They analyse and discuss tasks and problems related to a 
specific theme. The aspects that are associated with the theme originate from different 
disciplines. A theme covers a period of six weeks (with some units of four weeks to 
accommodate the fixed length of an academic year). During these blocks or units several 
other learning activities are organised such as skills training, practicals, lectures, practical 
medical training in health care settings. All these activities are related to the theme of the 
block and support the learning activities in the group sessions. 
Self-directed learning is an important element in the Maastricht problem-based pro- 
gramme. Therefore, the time scheduled for programmed activities is carefully limited, so 
that self-study time is available for the students. As a rule of thumb not more than about 
20 hours a week are devoted to scheduled activities. 

Requests for reprints should be sent to Hetty SneUen-Balendong, University of Limburg, Department of 
Educational Development and Research, P.O.B. 616, 6200 MD Maastricht, The Netherlands. 
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For each block-period a block book is compiled. The content of a block book has a fixed 
structure: 

- A general introduction which elucidates the block theme and justifies the activities 
and subjects chosen for tasks and problems. The relationship of the block to other 
blocks is clarified, and the global objectives are described. 

- A timetable in which each activity is explained. 

- The tasks and problems to be used in the small-group sessions, approximately 
twelve in each block. 

- A list of references to the literature related to the theme, and a list of other relevant 
learning-resources. 

- Self-assessment-questions. 

A fixed number of nine different disciplines are linked to each block theme. Five members 
of these disciplines form, within a rotation scheme, a group that plans and manages the 
block: the planning-group. From each of the nine disciplines two tutors are provided to 
guide the tutorial group sessions. There are about ISO students in each year of the 
programme, so that 1 8 small groups work simultaneously on the same tasks and problems. 
For each block the students and tutors are randomly assigned to their groups. 

The themes of the blocks in the first four years of the curriculum are set out in Table I. 
The Small Group Session 

With the exception of the first and the last session in a block, two tasks or problems are 
addressed in a single session. The session starts with a presentation and discussion of the 
outcome of the individual learning activities that resulted from the previous session. Any 
outstanding questions are explained as far as possible, and the task from the previous 
session is completed. A new task is then addressed and systematically worked through, 
until the group is able to define learning objectives for individual study in preparation for 
the first part of the next session. 

This systematic approach consists of the "seven jump" (Schmidt, 1 983), and this seven step 
procedure is outlined in key words in Table II. 

The most important task of the tutor is to facilitate the leanung process of the students. This 
implies that the tutor actively fosters the group's interaction* and discussion and interferes 
only when strictly necessary. Each student in rotation acts as discussion leader. For each 
session one student is appointed to record the discussion on the blackboard or on a flipchart. 

The Format of the PBL Material 

All the tasks and problems that are used as learning material in the small group sessions 
are included in the block book. The average problem requires less than half a page of print. 
For a particular problem the students are provided with sufficient information to arouse 
their interest and to motivate them to obtain more information and knowledge to 
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understand the background and the several aspects related to the problem. Sometimes the 
tutor is required to supply more information (for example, the results of a laboratory test). 



Table II. The seven jump approach 



Step 1: 


Clarifying whatever is not clear in the task. 


Step 2: 


Defining the problem. 


Step 3: 


Analysing the problem. 


Step 4: 


Listing possible explanations. 


Step 5: 


Formulating learning objectives. 


Step 6: 


Looking for additional information between group sessions. 


Step 7: 


Reporting back at the next tutorial group session. 



In most blocks the tutors have a tutor-guide. The educational objectives of each task are 
described in the guide, and questions are suggested that can be used to stimulate in depth 
discussion of the problem. Supplementary information, e.g. about a patient, may also be 
provided. 

The use of printed tasks in block books implies that the tasks are known to the students 
before the group sessions. However they have no information about the tutor guide. In the 
daily practice of the learning process students hardly ever study in advance. Perhaps this 
is the result of the way they are trained to work in a problem-based learning setting, but 
more likely this attitude is related to the fact that they are not assessed during the small 
group sessions. 

There is a difference between the tasks in the first two years and the third and fourth year 
of the curriculum. In the first two years a phenomenon or limited problem is the dominating 
task: a brief description of an event without referring to actual patients, their background 
or case history. In the following two years the problem of a real patient is the dominant task. 
An example of each type of task, together with its tutor guide is set out below. 
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A task from '^Locomotion", second block, Year 2 
Respiratory Control 

Situation 1 

A group of youngsters try to swim as far as possible under water. Harry can cross the pool 
twice (25 m length), and now Henk attempts to excel Harry' s performance. In order to 
increase the breathholding time he hyperventilates until he feels a little dizzy and then 
dives. He crosses the pool twice and makes a turn for his third lap. Half way through the 
third lap he suddenly stops swimming. Air bubbles reach the surface. Arno immediately 
dives into ae water and brings Henk to the side. Henk is unconscious, has stopped 
breathing and his heart rate is 98/min. Arno starts mouth to mouth resuscitation. After 
about one minute Henk starts to breath again and regains consciousness. His first question 
is: "What happened, and did I break the record?" 

Situation 2 

A 67-year old, dyspnoeic patient with chronic respiratory insufficiency is admitted to the 
hospital. The arterial pC0 2 is 8.6 kPa, the 0 2 saturation 70%, and the pH 7.28. The doctor 
administers oxygen at a rate of 2 liters per minute. The ventilation and the dyspnoea 
decrease, the arterial 0 2 saturation drops to 65%, the pC0 2 is 8.9 kPa and the pH is 7.18. 
The doctor wonders why the patient's condition is deteriorating. 



Tutor Guide 

The two cases describe two situations in which respiratory control plays an important role. 
After completion of this task the students should be able to explain what happens and 
should also explain the clinical findings. The control of respiration should be studied 
(peripheral and central chemoreceptors, respiratory centres and respiratory control). 
In Case 1 the hyperventilation decreases pC0 2 , but fails to increase p0 2 . During underwater 
swimming the decrease of p0 2 may induce unconsciousness before the pC02 stimulates 
breathing. 

In Case 2 the patient's ventilation decreases because of C0 2 intoxication, and lack of 
respiratory stimulation by too high an 0 2 delivery. 

A task from "Abdominal Complaints", second block, Year 4 

Mr. Smeets is 47 years old and has been a truck driver for the past 1 7 years. He used to work 
shifts, but stopped doing so when he developed a gastric ulcer. At present he works during 
the day, still smokes a lot and comes to his GP for stomach tablets at least 2 or 3 times a 
year. 
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Last week he was suddenly told to take a load to southern Germany. On the autobahn, half 
an hour after lunch, he developed severe abdominal pain. He just managed to reach a 
service area, where he had to vomit and where he began to transpire profusely. He rested 
for an hour and took two tablets of acetyl salicylic acid for the pain, because he had 
forgotten his stomach tablets. When he arrived at the hotel in Munich, he had to vomit once 
more. In the vomit he saw a bit of blood. He became alarmed, and took a taxi to the casualty 
department of the St. Franziskus Hospital. Following emergency endoscopy, the attending 
specialist told him he would probably need to have a stomach operation in the near future. 

Tutor Guide 

This task gives an example of a patient with an ulcer. Some causative factors should be 
discussed: stress, smoking, irregular meals, self medication with ASA. 
The ignorance of the patient regarding the danger of this medication for people with ulcers 
is to be stressed. 

This patient is an example of two possible complications: penetration/perforation and 
haemorrhage. The specialist who performed the endoscopy indicates a third complication: 
the danger of stenosis at the pyloric portion of the stomach. 

The choice between endoscopy and radiology for the diagnostic process in cases of 
diseases of the stomach are to be discussed. 

Regarding the therapy for this patient, the differences between intermittent ulcer treatment, 
maintenance treatment and surgical alternatives (elective resection of the stomach and/or 
vagotomy) are to be discussed. 

The discussion about the role of the eradication of Heliobacter pylori is optional. 

In the first task respiratory control is the key issue, and students are not supposed to discuss 
patient management. It is, therefore, important that the task is constructed in such a way 
that the students are not tempted to do so nevertheless. In the second example in the block 
"Abdominal Complaints", patient management is the most important issue, together with 
the pathophysiological aspects of this task. In this type of task it is important that the 
students do not jump to conclusions by considering the task as completed when they have 
determined the diagnosis. Dealing with patient problems, the SOAP approach (what are 
the Subjective elements we want to know about the patient, which Objective data are to be 
collected,howdoweEvaluatetheinfon^ 

over the "seven jump approach". However many groups intertwine elements of the seven 
jump approach with the discussion when pathophysiological topics are addressed. 
Materials other than the written information provided in the block book are seldom used. 
Sometimes a radiograph is made available for each of the groups. Occasionally, students 
are given the opportunity to see a video tape of a patient similar to the one presented in the 
problem. 
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Learning Issues Identified using Standardised Patients in a Problem- 
Based Learning Course 

Chris Eagle, Penny Jennett and Henry Mandin 
University of Calgary 
Calgary, Alberta, Canada 

Abstract 

The University of Calgary has a three year problem-based medical curriculum. Prior to 
starting the third year, students participate in a four week Integrative Course (IC). This 
course allows students to develop skills in counselling and managing multidisciplinary 
problems. In previous years, the IC has used patient management problems in computer 
and booklet formats. This year, eight professionally trained Standardised Patients (SP) 
were introduced to develop students 9 communication skills. This enabled the authors to 
study the nature of Learning Issues (U) identified by the students. During the pre-course 
orientation, preceptors were instructed to record all U identified by the students. These 
LI were then analysed by content area, learning category, and by resources used. Most LI 
concerned simple knowledge issues, most frequently pathophysiology, and these were 
resolved from textbooks. When comparing students 9 II with those of the case authors, 
groups of students identified an average of 38% of U overall. When U were divided into 
categories of basic science, diagnosis, pathophysiology, and management and counselling, 
students identified 42% of case authors 9 management and counselling issues. 

Introduction 

The University of Calgary Faculty of Medicine has a three year undergraduate curriculum. 
The first two years are composed of two parallel tracks. The first track comprises a series 
of body systems- based courses, for example, the respiratory and cardiovascular systems. 
The second track the Continuity Courses, focuses on skills and attitudes, and includes, for 
example, courses in human growth and development. The final year is a clinically based 
clerkship. The overall layout of the curriculum and the position of the Integrative Course 
is shown in Figure 1. 

There is considerable variation from course to course in the method of teaching. Overall, 
twenty per cent of the curriculum is in the format of Problem-Based Learning (PBL); 
thirty-five per cent is traditional small group tutorials; and the remaining fourty-five per 
cent is lecture format. : 



Requests for reprints should be sent to Chris J. Eagle, The University of Calgary, Faculty of Medicine, 1403- 
29 Street N.W., Calgary, Alberta T2N 2T9, Canada. 
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Calgary Medical School Curriculum 



First 2 years 



3rd year (clerkship) 



1st 
year 



2nd 
year 



PRINCIPLES FOR MEDICINE 
COURSE (4 weeks) 



REPRODUCTION 



RETICULO- 
ENDOTHELIAL 



PLANNING 

PRE- 
VENTION 



INFANT 



CHRISTMAS 



CARDIO- 
VASCULAR 



RESPIRATORY 



RENAL 



MUSCULO- 
SKELETAL 
& SKIN 



INFANT 



CHILD 



MIND 



HOLIDAY (3 weeks) 



ELECTIVE (4 weeks) 



NEUROSCIENCES 



ENDOCRINE 
METABOLIC 



FAMILY 



CHRISTMAS 



GASTRO- 
INTESTINAL 



AGING 
&THE 
ELDERLY 



INTEGRATIVE COURSE 
(4 weeks) 



EXAM* 

Clerkship 



September 



MANDATORY ROTATIONS 
-MEDICINE 
- PAEDIATRICS 
* SURGERY 
-OBSTETRICS A 

GYNAECOLOGY 
-PSYCHIATRY 
-FAMILY MEDICINE 
ELECTIVE TIME 



3rd 
year 



June 



April 



HOLIDAY (2 weeks) 



CHRISTMAS 



HOLIDAY (2 



UNIVERSITY EXAM 



MCC QUALIFYING 
EXAM 



May 



April 



Figure 1 . The University of Calgary Medical Curriculum 
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Students encounter PBL in the first weeks of the curriculum and recurrently into the 
clerkship. PBL at the University of Calgaiy follows the approach described by Schmidt 
(1983). Students, in groups of seven or eight, are presented with common clinical problems 
which they are expected to probe to the extent of their ability. The tutor' s role is to facilitate 
group process, challenge the student to exert their maximum effort, and to h«?lp identify key 
learning issues. 

The Integrative Course (IC) was developed eight years ago to help students improve their 
skills in dealing with multidisciplinary problems. During this four week problem-bated 
course, which occurs immediately before the clerkship, preceptors meet with studer.ts 
daily for two to three hours. The course uses a number of ill-defined problems of various 
formats including bock and computer based scenarios, and real and standardised patients 
(Barrows, 1987). 

This course has three, equally important, overall objectives: a) to develop clinical problem- 
solving skills (including history taking and physical examination); b) to impiove 
communication skills by exposure to problems which raise counselling issues; and c) to 
foster judicious use of resources. Counselling issues are a subtype of learning issues whose 
key feature is the process of communication, for example, how best to express key ideas 
or attitudes to a patient or coworker. 

In their orientation session and course documentation, the students are encouraged to 
explore the difficult issues posed by these patients. Many of the issues, for example death 
and dying and alcoholism, have been met in earlier parts of the curriculum during the 
Continuity Courses. In the Integrative Course, the students are encouraged to explore these 
problems in an interactive fashion with standardised patients. 

The tutors in this course are from a variety of medical backgrounds and are an equal mix 
of generalise and specialists. All have extensive experience in PBL, and most have been 
involved in the course for several years. Because the students have completed their final 
systems based course (Fig. 1), the tutors are instructed not to emphasize pathophysiology, 
but to encourage their groups to explore the deeper social, counselling and management 
issues. 

Over the years, the IC Committee had noted that the emphasis of the course had become 
weighted toward clinical problem-solving. In order to reemphasize management and 
counselling, eight new standardised patients were introduced. These were based on actual 
clinical cases and were developed by a case author (a physician) and a professional actor 
trainer. Table I shows that these cases were frequently multidisciplinary in nature and 
offered many potential counselling opportunities and complex management and counsel! ing 
issues. 
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The course committee wished to monitor how students would employ these new cases, and 
whether Learning Issues (LI) could provide useful information about the use of these cases. 

Table I. Standardised patients 



Case Initials Subject Matter 



1. 


T.B. 


Poorly controlled diabetic who wishes to become pregnant 


2. 


A.C. 


Patient with severely injured leg who refuses amputation 


3. 


L.C. 

• 


Immigrant woman with dysfunctional uterine bleeding 
who is concerned about the marital consequences of 
* hysterectomy 


4. 


J.G. 


Terminal care of a patient with metastatic melanoma 


5. 


E.H. 


A patient with unstable angina and lung cancer 


6. 


L.C. 


A patient with neurological deficits following a 
subarachnoid haemorrhage 


7. 


G.W. 


Alcoholism and hepatitis in a health care worker 


8. 


J.S. 


Adolescent with seizures, hyponatraemia, and 
anorexia nervosa 



Methods 

Prior to starting the course, a tutor guidebook was constructed which listed the case 
author's LI and the anticipated flow of the case. Meetings were held with the tutors to 
inform themofthechangesin the course and to solicittheir opinions aboutthe appropriateness 
of the management and counselling issues. Prior to the course, each standardised patient 
was reviewed by the case author and a senior resident to evaluate the accuracy and realism 
of the portrayal. In two cases substantia) changes were made to the standardised patient 
following this review. During the course, consistency of portrayal by the actors was tested 
by repeated, daily observation by the trainer and the course organisers. Informal discussions 
were held daily with the tutors about the actors 9 performance. Most of the actors had at least 
some theatrical training and all were working part-time in professional theatre. 

Case tutors were instructed to record all the LI identified by the students. The students were 
asked to identify LI whenever they felt more information was required to counsel the 
patient or to manage the particular situation. This was not limited to the student dealing 
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with the patient. In most instances, LI were recorded after group discussion which helped 
to define further the specific problem. Most of the sessions were also videotaped for 
subsequent review. 

The tutors were also asked to record the nature of the learning resources which the students 
used when they attempted to resolve specific LI. These resources included faculty 
members, texts, journals, computers, other library facilities, and outside agencies. When 
several students in a group used the same type of resource, for example textbooks, this was 
identified as a single learning resource. 

These LI were then analysed by content, by type of learning resource used, and by learning 
category. The details of this scheme are shown in Figure 2. The LI were also compared with 
those de vel oped by the case author for each category, and for management and counselling 
specifically. 

There were 12 groups of students, and a total of 18 tutors. Six tutors remained with their 
groups for the full four week block. The remaining tutors spent two weeks with their 
respective group. 



CONTENT 
AREAS 



Management 
Therapeutics 



Diagnosis 
Basic Sciences 



Pathophys. 




LEARNING 
CATEGORY 




Figure 2. Classification of learning issues 
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Results 

A total of 288 LI were compiled from 33 sessions with standardised patients. Typical data 
are shown in Table II. The percentage distribution of these learning issues, analysed by 
content area, learning category, and learning resources used, are displayed in Figures 3 to 
5. An example of a LI related to pathophysiology was: "How does hypothermia predispose 
to cardiac dysrhythmias?" From the same case, a management or counselling LI was: 
"How can informed consent be obtained for surgery, when the patient is hypothermic?" 
When the students' LI are compared to those of the case authors, the groups of students 
identified an average 38% of the LI overall, and 42% of the management and counselling 
issues. 



Percentage 




Basic Sciences Diagnosis Pathophysiology Therapeutics Management 



Content 



Figure 3. Categorisation of learning issues by content area 

The method used by the groups of students to identify LI was not studied specifically, but 
from direct observation and review of videotapes, it was apparent that potential issues were 
identified by one student or simultaneously by several students. This was followed by 
group discussion about the topic, and if agreement was reached within the group, it would 
be recorded as a LI. Most frequently, the tutor was involved only in the later stages of the 
process. 

In general, students identified fewer LI than planned by the case authors and rarely 
identified issues that were not included in the authors' lists. As can be seen from Table II, 
there was variation between groups in the number of LI identified. There was a consistent 
tendency for students to identify primarily "knowledges-related topics. This tendency 
reflects the distribution of the case author's LI (Table II). 
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Percentage 



Knowledge 



Learning Categories (N= 288) 




Skill 



Attitude 
Category 



Figure 4. Analysis of learning issues by category of learning 
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Figure 5. Resources used in the resolution of learning issues 
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Some difficulties were noted in associating a specific LI with the corresponding LI of the 
author or between LI generated by different groups of students. Differences in terminology 
or the detail of recording could result in potential variation in the assignment of a particular 
LI to a given content area or learning category. For the majority of recorded LI, this was 
not a problem, but a greater potential for misassignment was felt to exist when recorded 
LI were extremely terse. 

There was a consistent pattern in the use of textbooks to answer questions related to 
pathophysiology and in the use of teachers for questions related to management and 
counselling (Fig. 5). Students occasionally used multiple approaches to answer * single LI; 
for example, one student would attempt to answer a question from a book, a second would 
use a Faculty member, and a third would consult an outside agency. The first part of the 
group's next meeting would be spent synthesizing this information. 

Only one significant problem was noted in the consistency of portrayal by the actors. This 
was resolved by changing an actor. At the debriefing of the tutors, no further problems were 
noted with the performance of the actors. 

Discussion 

It is apparent from this and other studies that the examination of documented LI can provide 
useful information about the content of learning sessions when standardised patients are 
used (Coulson, 1984; Shahabudin, 1987). Course managers can identify how students 
spend their time; which key issues are, or are not noted or discussed; and which learning 
objectives are addressed. For example, at their debriefing tutors noted a general weakness 
in students' physical examination skills as reflected in the lists of LI (Fig. 4). This 
information has led to greater emphasis on physical examination in the tutor guide book. 

An objective of the course committee was to increase the emphasis on management and 
counselling. Management comprised the second largest group of identified LI (Fig. 3). 
Since objective information from previous courses was not available, the authors could not 
demonstrate any change in the number of management LI. However, the opinions 
expressed by preceptors at the debriefing suggested that an increase in this type of LI had 
occurred. Most LI were based on pathophysiology. As these students had previously 
completed their systems based courses, the emphasis on'pathophy siology was interesting 
and may lead to further exploration. The end-of-course questionnaire revealed that many 
students used this course to review concepts from the system based courses. 

Although textbooks were the most commonly used learning resource, students used 
Faculty staff for almost Qne quarter of all learning issues. This practice was encouraged by 
the course committee. Resource persons were identified in the preceptors 9 case guidebook 
and were familiar with the relevant cases and objectives of this course. While one objective 
of the course committee was to encourage students to contact outside agencies (for 
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example, Alcoholics Anonymous), this was found to be an infrequent occurrence and 
indicated the need for further development. The use of computer or laboratory facilities 
was not actively encouraged or discouraged These resources were not specifically 
identified in the tutor guidebook. 

When viewed from the perspective of categories of learning, most LI were concerned with 
simple acquisition of knowledge. Very few LI were directed toward development of 
attitudes or skills, a deficiency which will be addressed by the course committee. 

The relatively low percentage of the case authors' LI that were identified by the students, 
requires clarification. The LI lists compiled by case authors are exhaustive and contain 
many issues already identified in the objectives of the previous systems based courses. 
Tutors used the authors 9 lists as guidelines, but did not expect that all issues would be 
identified. In attempting to change this course, more specific training of the preceptors may 
have resulted in greater attention to the management and counselling issues. Preceptors 
were led through each case by the author in the orientation session with ample opportunity 
for discussion. A more interactive training session that would allow the preceptors the 
opportunity to work with the standardised patients directly, may prove to be more 
effective. 

Conclusion 

In reviewing these PBL sessions, it was apparent that the review of LI is an extremely time 
efficient technique for course administrators, compared with direct observation and 
videotape analysis. Large numbers of sessions can be reviewed rapidly, and additional 
information can be obtained about work done by students in their own time. 

However, some limitations became apparent Recorded LI reflect neither the time spent by 
the group on a particular issue, nor those issues that had been resolved within the group. 
Direct observation and review of sessions on videotape indicated that considerable 
discussion of management problems may occur within the group without generating a 
single recorded learning issue. By contrast, items related to pathophysiology tend to be 
recorded quickly with smooth transition by the group to the next area of interest. This is 
a significant limitation in the use of LI for assessment of group activity. A further limitation 
was the occasional difficulty encountered in associating a specific LI with those generated 
by the case author, due to differences in terminology or the relative accuracy and 
, completeness of the recorded LI. This may hinder identification of a group* s actual activity 
or point of interest. 

We conclude that the review of learning issues that are identified while working with 
standardised patients can provide useful information for course managers, but the method 
has significant limitations. 
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Some Aspects of Skills Training and Problem-Solving in the Medical 
Faculty, Diponegoro University 

Bambang-Hartono, Soenarto, Imam-Parsoedi and Djokomoeljanto 
Diponegoro University 
Semarang, Indonesia 

Abstract 

The paper outlines an introductory course to clinical medicine which is presented over a 
period of eight weeks between the preclinical and clinical components of the undergraduate 
curriculum. Emphasis is placed on practice through role play among students. The use of 
a number of methods for learning and assessment is included in the paper. 

Introduction 

The introductory course to the General Clerkship links the theory and practice of medicine. 
It was felt that new clinical students were inadequately equiped to benefit from the clinical 
setting. There was also a need to bridge the change from theory to practice and from 
knowledge orientation to problem orientation in their learning. In addition, this course was 
to avoid overlap of introductory activities by various clinical departments. The course aims 
to provide training in basic medical skills, including obtaining data in history taking, 
physical examination and routine laboratory examination. It is also designed to refresh the 
students' knowledge and to make this knowledge applicable to medical practice (applying 
the process of problem-solving). The course comes after the first four mainly theoretical 
years and before the two year clerckship (Fig. 1). 



1st - 7th semester 
Mainly theoretical 
knowledge 



8th semester 
Introduction to 
Clinical Clerkship 



9th- 12th semester 
Clerkship 



Figure 1. Sequence in the curriculum 

The duration of this course is eight weeks and is designed as small-group tutorials for 
groups of 12 students (Tim Panum, 1989). The tutor guides each student and and assesses 
achievement of the goals. The tutors are trained in a special two week course. They are 
recruited from preclinical, paraclinical and clinical departments. 



Requests for reprints should be sent to Bambang Hartono, Diponegoro University, Faculty of Medicine, Jl. 
Nangka Barat No. 5, Semarang 50249, Indonesia. 
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History Taking 

1. The student is able to perform medical history taking and master interpersonal 
communication aspects adequately. 

2. The student is familiar with emotional reactions and resistance of a patient. * 
History taking is practised in role-play sessions with emphasis on communication as well 
as medical aspects. Special videotapes have been developed to demonstrate problems in 
doctor-patient communication for study before role-playing excercises are undertaken. At 
first, a list of topics is given to the students one week before the role-playing. These topics 
represent chief complaints to be used in the role-play. Each group of 12 students is supplied 
with 12 chief complaints, one for each student: 

1. Coughing 7. Haemoptisis 

2. Headache 8. Haematemesis 

3. Chest pain 9. Fever 

4. Abdominal pain 10. Weight loss 

5. Diarrhoea 11. Bleeding 

6. Back pain 12. Dyspnoea 

These are prominent chief complaints in rural and urban primary health care (PHC) in 
Central Java. Each student acts as phycisian as well as patient but with a different topic, 
so that every student has the same experience of skills training in history taking. During 
the role-playing sessions discussions are tape-recorded as evidence during discussion 
among the students. At the end of a session the tutor comments and offers suggestions on 
medical as well as communication aspects. 



Physical Examination 

1. The student is &*>le to execute the routine physical examination. 

2. The student is aware of the significance of undergoing and executing physical examination. 
At first students browse through videotapes of physical examination, prepared by our 
Audiovisual Division. The students will then practise physical examination in pairs, one 
acting as the doctor and the other as the patient alternately, guided and supervised by the 
tutor. 

For sociocultural, ethical and religious reasons, male and female students are trained in 
separate rooms. To strengthen the students* experience in the diagnostic process and its 
strategies (Djokomoeljanto, 1989), we show them a series of specific clinical pictures on 
slides and videotape. 

Routine Laboratory Examination 

1. The student is able to perform routine laboratory examinations in preparation for his 
activities on the ward. 

2. The student is aware of the significance of undergoing and performing laboratory 
examinations. 
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In this session students are trained in performing routine laboratory examinations through 
practice with each other, preceded by discussions that are guided by the tutor. 

Medical Problem-Solving 

The term "medical p!roblem-solving" is most often used to refer to that process by which 
physicians arrive at a diagnosis, select a management strategy, initiate remedial action, 
monitor its effectiveness, and modify their interventions accordingly. The situation in 
which this process may be used ranges from the problems of individual patients to 
community or national health problems (Kuyken et al., 1988). 
We use several m: Chods to introduce medical problem solving: 

1. Modified essay questions (MEQs) 

2. Pharmacotherapy 

3. Problem-oriented medical record (POMR). 
Modified Essay Questions (MEQs) 

We select three kinds of written simulation for problem-solving exercises. These are 
Problem Box, Patient Management Problem and MEQ. An MEQ is a case history, as it was 
obtained in practice. Information is provided in a sequential manner. The technique is in 
effect a series of open-ended questions about a developing situation in which the responses 
at each stage are relatively free. The students have to construct their own responses. On the 
next page of the MEQ, new information is given to continue the case (Feletti et al., 1980). 
In this developing situation of a given case, questions can be devised to explore important 
factors influencing decision making and attitudes, thereby gaining insight into the 
students' behaviour. Also some of the thinking processes leading to these actions can be 
demonstrated (Verwijnen et al., 1989). 

We have prepared more than 30 MEQs for use in medical problem-solving skills training. 
They have been designed by the Division of Problem-Solving in the Introductory Course 
team. Topics of MEQs are selected by considering prevalence in general practice. MEQs 
are discussed in small groups, guided by a tutor. 

Pharmacotherapy 

From the clinical situation, the student is able to reach a justified choice of drug therapy. 
The student refreshes his/her knowledge of therapeutics and pays attention to important 
interactions. The application of pharmacotherapy is here practised through the solving of 
written simulated patients in MEQs, but with emphasis only on pharmacotherapy. 

Problem-Oriented Medical Record (POMR) 

The student can demonstrate the relative significance of the essential problems that have 
been registered. The student is able to reproduce the principles of the S-O- A-P (Subjective- 
Objective-Assessment-Plan) system as the major framework of POMR (Imam-Parsoedi 
et al., 1988; Petrie et al., 1979). POMR training is implemented in small-group tutorials, 
by producing and filling in the POMR, followed by discussion. 
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Other Clinical Skills 

Several clinical skills are practised during the Introductory Course, including 
cardiopulmonary resuscitation, first aid and nursing. 

Assessment of Students 

Several kinds of tests are used, but mainly the Objective Structured Clinical Examination 
(OSCE). Here the students rotate through stations where they are required to perform a 
variety of tasks (Harden et al., 1979). They may be asked to perform a neurological 
examination of the lower extremities, an abdominal examination, etc. Their performance 
is rated by tutors on standardised checklists. 

Evaluation 

Experience over a period of three years has demonstrated: 

1. Improved clinical performance, especially in clinical skills and problem-solving. 

2. A more effective and efficient curriculum, because overlap in basic clinical training by 
separate clinical departments has been reduced. 

3. Better cooperation among the teaching staff from different departments, because they 
are closely involved in the planning and conduct of the Introductory Course. 

Conclusions 

The Course provides an Implementation Team that supports the Dean in the implementation 
of the curriculum, and a Research and Development team which has to develop further the 
contents, methods, strategy and evaluation of the Course. The Course has demontrated the 
advantages of applying the problem-based learning principles. We are of the opinion that 
we will achieve more if more problem-based learning activities can be implemented in our 
medical education. 
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ADMINISTRATION OF EDUCATION 

A Field Experiment on Admission to Medical Education at 
Linkoping University 

ils-Holger Areskog, Carl Holmberg, Sture Ulfberg and the Admission Steering Group* 
Linkoping University 
Linkoping, Sweden 

Abstract 

The paper describes an individualised admission procedure to medical education at the 
Faculty of Health Sciences, Linkoping University. The main goal for the procedure is to 
admit students who will be successful in a problem-based learning course and who can 
be expected to become good professionals. The admission procedure consists of three 
steps. Step I is the conventional ranking of applicants, based on marks from upper 
secondary school or on the university standard aptitude test. Step II consists of three tasks: 
written explanation of motivation, structured personal life description and an essay. Step 
III consists of an interview conducted by a pair of interviewers, one layman and one 
academic practitioner. Figures from two admissions showed acceptable reliability in Step 
II and very good reliability in Step III. An inquiry among applicants revealed good 
confidence in the competence of the interviewers. The procedure will be used for three 
years and it will be evaluated both on a sh^ '-term and a long-term basis with regard to 
reliability and validity of the admission instruments. 

Introduction 

The Medical Faculty at Linkoping University was established in 1970. Until 1986 its 
medical students spent their first two (preclinical) years at the University of Uppsala and 
the last 3.5 years in Linkoping. The entire study programme was a conventional medical 
curriculum. From the Autumn 1986 the Faculty of Health Sciences was established with 
curricula for six different health professions. Problem-based learning (PBL) was adopted 
as a common educational principle. In the medical curriculum training in patient 
communication and examination starts in the very first semester. Multiprofessional 
education (MPE) fot the six health curricula is introduced during the first ten weeks and 
intermittently thereafter. Other key-concepts are preventive medicine and primary care . 
The first medical students in the new curriculum graduated in January 1992. 

* Berit Askling, Ricci Gotthard, lunnel Norrby, John-Erik Ottos:;on, Monica Qvicker- 
Andersscn 
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Admission to all full-length study programmes at university level is ceu-alised in Sweden. 
A special unit at the National Board of Universities and Colleges deals with admission. 
Students who wish to apply to any or all six medical schools address themselves to that 
board. Applicants have to declare an order of priority. The criteria for admission vary, but 
the main discriminant is grades from secondary school or marks from a university standard 
aptitude test (US AT). Everyone above school-age can take the test With effect from 1991 
every second and third year student in upper secondary school can also take this test and 
use it as an alternative to their school grades. At present 60% of the students are admitted 
to medical schools on their school grades and 40% on their USAT marks. The academic 
year in Sweden has two semesters, and new students are admitted twice a year. 

When the new curriculum was first was designed in Linkoping, a more individual 
admission system was included in order to admit students with a favourable prognosis in 
the completely new curriculum. The proposal was repeated by the Faculty of Health 
Sciences but was rejected by the Government, mainly because of the methods that had been 
suggested. Interviews were not looked upon as a "scientifically proven tool, adequate for 
university admission". 

However, from 1990 onwards the political climate became more favourable for local 
initiatives in university admission. A new proposal from Linkbping was now approved. 
The proposal encompassed a three year experiment with an individualised admission 
system for the educational programmes for physiotherapists and medical doctors, to be 
followed by extensive evaluation. 

This paper describes the principles of the individualised admission system for the medical 
programme and our experiences after two admissions. 

Reasons for the New Admissions Procedure in Linkoping 
Medical education in Sweden attracts many applicants. Very few arc accepted into a 
medical school. Almost all are academically among the best in their age-group. The 
problem is not to find the brightest students, but to find the students who are most likely 
to be successful in their studies and in the profession. 

When students enter the educational system at the Faculty of Health Sciences in Linkoping 
they meet something that is quite new for most of them. It is, therefore, very important that 
no one should enter the course without being fully aware of what is involved. 

The main goal for the selection is thus to identify students who are: 

* well informed about the study system in LinkSping; 

* successful in an unconventional curriculum with e.g. problem-based learning (PBL) 
and multiprofessional education MPE; and 

* expected to become good professionals. 
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Overall Design of the Experiment 

The experiment will last for three years with careful follow-up of different effects. During 
the three years there will be six admissions. Two have already been completed. On each 
occasion half of the students to be admitted will be selected by the local procedure/The 
other half will be chosen by the central authority according to normal procedure. The 
treatment variable is here the different admission system. The first group consists of 
approximately 18 students per semester, but this will increase to 23 per semester from 
Autumn 1992. The same numbers apply for the second group, the control group in the 
experiment. The effect variables are related to success or failure for the different groups 
on the goals stated for the local admission procedure. 

The procedure for the experimental groups consists of three steps. 
Step I: All applicants are ranked according to their school-marks and/or USAT scores. 
From the group with top marks approximately six times as many as will finally be accepted 
are chosen for the next step. There are two restrictions. Only those students who have 
elected Linkoping as their first priority can go to the next step, and only those who have 
declared their willingness to participate in the admission procedure at Linkdping. 
Step II: A couple of months after their application has been received, around 100 
applicants come to Linkoping for almost a full day. They will receive extensive information 
about the educational system at the Health Faculty and undertake different written tasks 
(see below). The 45 best of these applicants proceed to the next step. 
Step HI: One to two weeks after Step II these 45 applicants come to Linkoping for a one- 
hour interview (see below). A final ranking of the applicants results in the top ones being 
admitted. 

Although the control group is admitted according to the national admission rules, these 
students are interviewed during the first days of the course in the same manner as the 
experiment group was during Step III. 

A student who accepts to take part in the local admission procedure in Linkoping does not 
loose his or her place in the ordinary national admission process for medical schools, but, 
where two offers are made, the admission to Linkoping takes precedence. 

The effect of the two admission procedures will be investigated by collecting data about 
student drop-outs, and successes and failures in examinations. Data that may indicate the 
prognostic value of the admission procedures are obtained from written, oral and practical 
examinations after every semester (20-21 weeks) during the whole study programme. At 
the end of semesters three, five and eleven more complex examinations take place. These 
include video-taped patient interviews and examinations, problem identification and 
problem solving, survey of the literature, and presentation of actual research data 
highlighting a relevant patient problem. 
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After Steps II and III the applicants and everyone who has been involved in the selection 
programme participate in an evaluation of the arrangements, the administrative procedure, 
and how they have experienced the admission procedure as a selection instrument. The 
reliability of the admission instruments is calculated after every admission, and some data 
are given below. 

Design of the Selection Instrument 

The selection procedures at the medical schools in Newcastle, Australia, Beer-Sheva, 
Israel, and McMaster University, Canada were studied both during site visits and from the 
literature (Antonovsky et al., 1979; Vinson et al., 1979) . In the literature there are 
conflicting reports regarding the value of the interview for the selection of medical 
students. DeVaul et al. (1987) found that the interview of the Houston model lacked 
predictive value in relation to medical career, and the method had an adverse cost-benefit 
ratio. However, these results conflict with those at the University of Newcastle, Australia 
(Powis et al., 1988), where the interview identified students who failed to complete the 
course and helped to predict students who graduated with honours. 

There is support in the literature (Mischel, 1968; Magnusson, 1981) that concrete action 
in contrast to general verbal expression of one's will makes prediction of future behaviour 
possible. Therefore, the instruments were based on the principle that the best way to gain 
an idea of what you may be doing tomorrow is to ask what you are doing today. 

Special selection instruments have been constructed for Steps II and III. In Step II written 
tasks are evaluated by a pair of assessors who are teachers in the Faculty. In Step III 
interviews are conducted by an academic and a layman of the opposite gender. The 
instruments were designed to generate data on aspects of the personalities of the applicants. 
The variables can be summarised under four categories: a) General study ability andcognitive 
flexibility. These aspects are highlighted in Step I. The assessment in Steps II and HI 
concentrates on abilities of special importance for both the study of medicine and the 
medical profession; b) Motivation. Overall interest for medical education is evaluated. The 
reasons for chosing to study medicine are scrutinised, and the degree of realism of the 
candidate" s view of the profession is assessed; c) Mental energy. Endurance, stress tolerance, 
decision making, and ability to overcome difficulties and misfortunes are examined; and 
d) Social competence. This category reflects the candidate's ability in various situations 
to relate to patients, family, colleagues, and professionals from other professions. It has 
many facets of special relevance to health care. Sub-characteristics are, e.g. communication, 
collaboration, management, self-evaluation, tolerance, and integrity. 

These categories and their characteristics form the basis for the design of both the written 
tasks in Step II and the content of the interview in Step HI. 

Step II includes four sessions. In the first the applicants obtain information about the design 
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of the special study-programmes at Linkoping. The following three sessions include a 
written explanation of motivation for the choice of this professional career, a structured 
personal life description and an essay. The whole procedure takes about five hours 
including a break for lunch. 

The written explanation of motivation is primarily designed as a basis for evaluation of 
motivation (b), but the design of the tasks makes it possible to evaluate also mental energy 
(c) and social competence (d). 

On a preprinted proforma the applicant can highlight his/her motives for medical studies, 
especially the Linkoping model, and his/her choice of occupation in a maximum of 300 
words. 

A structured personal curriculum vitae provides a basis for assessment of mental energy 
(c) and social competence (d). The applicants are informed that their experiences must be 
confirmed by referees. 

Social competence (d) (linguistic ability and personal flexibility) is evaluated in the essay. 
In three to four pages the applicants have to deal with one of three themes. The task is to 
highlight the theme with as many relevant aspects as possible and to discuss it. Examples 
used so far are: "Private or public health care delivery?" and "Controversy about free 
abortion". 

Step III consists of an interview. During the interview a final evaluation of the characteristics 
(b), (c) and (d) is made; special emphasis is placed on social competence (d). All interviews 
are opened and closed in the same way, and a number of areas are explored by the 
interviewers. The interview is not otherwise structured. There are no fixed questions 
except for the introductory and the final ones, and there is no predetermined sequence for 
the different areas to be explored. Hie interview takes 45-60 minutes, and the two 
interviewers alternate in posing questions. 

The instruments for selection are confidential and they &re retained by the admissions 
committee. 

The Selection Procedure 
StepO 

In order for Step II to be as objective as possible, assessment manuals were developed for 
the written tests. The assessors are teachers in the Faculty (mainly physicians) and work 
in pairs. The applicants are divided into groups of approximately 20. The groups are 
matched for age, school marks or scores at US AT and randomised.The groups are allocated 
at random to the evaluators. The papers are assessed by the two assessors independently 
of each other. Subsequently they meet and decide a common ranking order for the 
applicants within the group. The assessors also supply the admission committee with a 
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report on the overall quality of the group. If the assessors do not reach consensus, separate 
ranking is given to the admissions committee with reasons for the disagreement. However, 
this has not happened so far. 

In both Step II and Step HI the performance of the applicants is ranked on an ordinate scale 
from 1 (bad, unsuitable) to 5 (excellent). From every group the nine applicants with the 
highest ranking are invited to participate in Step HI. 

stepm 

The interview is conducted by two interviewers - one male and one female. One comes 
from the Faculty of Health Sciences, and the other is a layman with experience in assessing 
human qualities, e.g. a judge, lawyer, priest, teacher. Groups for interview, 9-10 students, 
are composed in the same manner as for Step II. The interviewers rank all applicants 
independently before they arrive at a common rank order. In case of severe disagreement 
on an applicant (>2 scale steps in the overall rating), the applicant will be interviewed again 
by another pair of interviewers. During the first two admissions this has not happened. The 
admissions committee undertakes the final selection, based on the interview data. 

Instruction and Training of Assessors and Interviewers 

The interviewers were given a whole day of instruction and training, and the assessors were 
given half-a-day. Both groups received written guidelines for the ranking of the applicants 
in order to limit differences between interviewers or assessors and between the interviewer 
or assessor teams. 

Organisation for Planning and Implementation of Assessment and Selection 
The responsibility for the planning and implementation of the new selection procedure was 
given to a Steering Group selected by the Board of the Faculty of Health Sciences. The 
development of the special selection instruments was undertaken by the Department of 
Education and Psychology, Linkoping University. They were responsible for: 

* defining selection criteria; 

* developing special selection instruments; 

* constructing assessments in Step II, checklist, and training of Step II assessors and Step 
HI interviewers; 

* planning of the first two selections; 

* acting as consults during the implementation of the admissions procedure; 

* controlling the reliability of the selection instruments; 

* estimating time, costs, etc.; and 

* discussing principles for the long-term evaluation. 

Before the instruments were used in practice they were tested with upper secondary school 
pupils. The final design was decided by the Steering Group. Test interviews were video- 
taped and discussed by the Steering Group, in order to establish more precise criteria and 
proposals for interview questions. 
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The Steering Group was also the admissions committee for the first two admissions, but 
a specially elected admissions committee will thereafter assume that responsibility. That 
admissions committee will consist of the director for undergraduate studies and 
representatives of the teachers and students. 

Experiences from the Autumn 1991 and Spring 1992 Admissions 
A large number of students applied on both occasions, 925 for the Autumn semester 1 991 
and 405 for the Spring semester 1992. From these groups 17 and 18 applicants respectively 
were admitted as the control group according to their marks from upper secondary school 
or test scores in the USAT. Of the 925 applicants 740 satisfied the condition that they had 
nominated Linkoping as their first choice. For the Spring semester, of 405 applicants 3 14 
met these restiictions (nominating Linkoping as their first choice). 

For the local admission selection the figures for the different steps were as follows; the 
figures for Spring 1992 are presented within brackets. In Step II 1 1 8 (100) applicants were 
selected from 740 (3 i 4) who had nominated I ,inkoping as their first choice; 46 (45) of these 
applicants proceeded to Step III; and 18(18) were admitted following the interview (Table 

i). 

Table II shows the age distribution at the different steps. The tendency that the older 
applicants were favoured for the Autumn entrance was not significant (p = 0.09). 

The reliability of the methods was studied as a percentage agreement (A) directly related 
to the score difference between two assessors (interviewers) in the scoring of different 
characteristics: 

Sum of score differences 

A = 1 - x 100 

Maximum number of score differences 

(Ost, 1979) 

Sum of score differences: The sum of all the differences between two assessors. Maximum 
number of score differences: The sum of possible differences between two assessors. The 
results for Step II (Table III) are all significant (p < 0.0 1 ) . Table IV shows the corresponding 
agreement between two interviewers for five interview groups in Step m. Ail are 
significant (p < 0.01). 

The applicants and the assessors in Step II and the applicants and the interviewers in Step 
III answered a questionnaire. The instrument called for the use of rating scales and 
comments. The scales had seven steps from bad, incomplete, etc. (1) to good, complete, 
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etc. (7). The applicants rated different aspects of the admissions procedure and how they 
estimated their opportunity to demonstrate their aptitude for the study of medicine and their 
potential success as a doctor. The response rate for the applicants was 1 00%. The assessors 
and interviewers rated different aspects of the admissions procedure and their opinion of 
the reliability and validity of the instruments. At each step two did not complete the 
questionnaire. 

The applicants recorded very positive ratings for almost every aspect of the admissions 
procedure on both occasions and in both Step II and Step HI 6.4±1 .2. Their ratings on the 
face validity of the admissions procedure demonstrated considerable confidence in the 
instruments. In Step II their rating of the opportunity to demonstrate their potential 
educational and professional success and their motivation for their studies and profession 
were 5.2±1 .4. In Step m the equivalent ratings were 5.8±1 . 1 . In Step HI the applicants also 
expressed their confidence in the interviewers and rated their competence. These ratings 
were very high, 6.4±0.9. These statistics represent weighted means and standard deviations 
combined from several items. 

On the whole, the assessors and interviewers were also very positive towards the 
admissions procedure. They contributed on average 14hoursinStepIIand 18hoursinStep 
III. When they commented on, or rated different aspects of reliability, they demonstrated 
a fairly strong belief in the instruments. However, they were not so sure about a strong 
relationship between the admissions procedure and the main criteria (academic and 
professional success). 

Discussion 

At the first selection for admission there was a very large number of applicants. On the 
second occa sion the number was considerably smaller. The reason was probably an urgent 
need for this type of admission for applicants who arc less successful in upper secondary 
school or US AT examinations, resulting in a high number of applicants with lower scores. 
On the second occasion there wis a more normal number of applicants who gave 
Linkoping as their first choice. The figures from the two admissions confirmed earlier 
experiences in Sweden that grades from upper secondary school favour girls and US AT 
results favour boys. 

Table I illustrates that the girls obtained lower scores in the aptitude test and, therefore, 
decreased considerable the numbers between Step I and Step II. However, more girls than 
boys were selected, based on marks from upper secondary school. This difference is 
statistically significant p<0.01 (chi-square) for both admissions. The figures for the 
controls show that school examination marks clearly favoured girls and USAT favoured 
boys. This observation is confirmed by a recent national study (H. Jansson, personal 
communication). 
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There was a clear tendency at the first selection that older applicants score more highly on 
life experience in Step n. Therefore, at the second selection the relative weight of these 
experiences was decreased, in order to lessen any bias between applicants above and below 
22 years of age. The change was marginal and did not in essence change the prerequisites 
for the experiment 

Tables III and IV show that reliability was acceptable in Step II and very good in Step III. 
The reason for higher reliability of the interview scores is probably that the interviewers 
cooperated better as they compared their respective rating after every interview. Also, the 
interviewers had a whole day of training, and it is probable that the assessors for Step II 
should have more than half a day of training. 

The enquiry among both applicants, assessors and interviewers showed good satisfaction 
with the administrative procedures and arrangements. The applicants also demonstrated 
confidence in the competence of the interviewers. 

There are, of course, no data as yet on the prognostic value of the admissions procedure. 
The evaluation will use indicators such as permanent or temporary drop out, counselling 
to change from medical studies, e.g. due to unsuitable personality, poor results in 
theoretical and practical clinical examinations and unsatisfactory assessment during the 
clinical training period. The students will contribute by reporting, during their evaluation 
after every semester, on the development of their learning process and self-development. 

Validity is the main issue for the whole experiment. So far we have two different views on 
the face-validity of the instruments. We are confident that concept-validity is profound. 
Psychological expertise has based the components in the different instruments on well 
established theories. 

The admissions procedure is quite costly compared with traditional practice. However, 
every permanent drop out is expensive for the Faculty. This will be important in the 
proposed new university system in Sweden where the Faculty will obtain resources in 
relation to the number of students who graduate. The costs of the admissions programme 
was around $ 800 for each student admitted (cost of staff time not included). This is 
acceptable in the light of the high expenditure for every medical student at approximately 
$ 18,000 per year for the five and a half year course. 
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Table III. The mean reliability as a percentage of the 5 pairs of assessors in Step II for the 
three tasks 

Autumn 1991 Spring 1992 





Mean 


Range 


Mean 


Range 


Life description 


82 


69 


-90 


84 


73- 


93 


Motivation 


70 


63 


-78 


77 


65- 


87 


Essay 


70 


61 


-83 


81 


76- 


88 



Table IV. The mean reliability as a percentage of the 5 pairs of interviewers in Step III for 
the four criteria. 



Autumn 1991 



Mean 



Social competence 93 

Psychic energy 96 

Motivation 91 

Summarised 
evaluation of 
eligibility 94 



Range 

86-97 

92-100 

88-97 

90-97 



Spring 1992 
Mean 

95 
95 
91 

95 



Range 

89-97 

89-100 

81-97 

86-100 



Conclusion 

This form of selection for admission is new for Sweden. The results are based on only two 
of the six selection exercises in the present experiment. 

In contrast to earlier negative attitudes by government, central authorities, academies and 
students to local and individual admission systems, the present experiment has so far been 
an overwhelmingly positive experience. The selection instruments for Steps II and HI have 
functioned well and are regarded with confidence by both applicants, assessors and 
interviewers. The experiences are still too limited for an evaluation of the validity of the 
instruments. The reliability of toe instruments has so far proved to be quite acceptable. 
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The Program for Educational Development 

Geoff Norman 
McMaster University 
Hamilton, Ontario, Canada 

Abstract 

The Program for Educational Developmen t ( PED) at McMaster University is now over 20 
years old. Its organisation was a radical departure from the conventional office of medical 
education. It is organised along the lines of an educational programme with academics 
recruited from a number of departments. The focus of the PED is on educational research 
and development, and its responsibility to the Faculty is primarily at the level of 
consultation and innovation. The approach has certain advantages and disadvantages, 
which are discussed. 

Organisation, Mission and Activities 

The innovative medical school at McMaster University entered its first class in 1969, and, 
shortly thereafter, the Program for Educational Development (PED) was established as the 
locus for educational research and development activities. The establishment of an office 
of medical education as an adjunct to the affairs of the medical school was not uncommon 
in those times. George Miller had begun the first office of medical education not far away 
at Buffalo, New York, in the mid 1960's, and this model had "metastasised" to the large 
and renowned offices at Michigan State University under Hilliard Jason, at the University 
of Illinois under George Miller, and at the University of Southern California under Steve 
Abrahamson. Vic Neufeld, the first chairman of the PED, spent a sabbatical year at 
Michigan State in 1969, and had the opportunity to view the operation of such an 
administrative structure at first hand. 



Neufeld, together with Howard Barrows and Jacqui Wakefield, chose a model for 
McMaster which was very different from these offices; a difference which is, in fact, 
embodied in the title. First, it is a programme, not an Office or a Department. At McMaster, 
these words have significant meaning. From the outset, the Faculty had adopted a "matrix 
management" approach, in which all education, research and service activities took place 
through programmes, while individual faculty members resided in departments. Program- 
mes have separate administrative structures from departments, and have no faculty 

Requests for reprints should be sent to Geoff Norman, McMaster University, Faculty of Health Sciences, 
Department of Clinical Epidemiology and Biostatistics, 1200 Main Street West, Hamilton, Ontario L8N 3Z5, 
Canada. 
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appointments. The prototype Programme was the undergraduate M.D. (Medical) pro- 
gramme, and the administrative structure was designed to ensure that the curriculum would 
not be dictated by the Chairmen of departments. PED, by becoming a programme, is more 
closely allied with other educational and research programmes, drawing support from a 
diversity of departments, than as a separate department with its own academic staff. To this 
day, no faculty member has an appointment in PED - the chairman is in th^ Department 
of Clinical Epidemiology and Biostatistics (CE&B), and other core members are based in 
the Department of Family Medicine or CE&B. 

"Educational", the second word in the title, is also significant. It is not "Medical 
Education". At McMaster, the medical programme is one of many educational programmes 
in the Faculty of Health Sciences. It was perhaps prescient on the part of the original 
architects of the PED to avoid the term "medical" in the title. It has provided scope for the 
involvement and substantial contributions by professionals from a variety of health 
disciplines (nursing, physiotherapy, occupational therapy, etc.). 

The last word, "Development", actually stood for "Research a'ld Development" in the eyes 
of the original planners. PED certainly has maintained these parallel activities throughout 
its history. Development has always been there, but it has been accompanied by an 
emphasis on exploring the 4< basic science" of education, and PED members have contributed 
substantially to theoretical and methodological developments in the study of clinical 
reasoning and in research into evaluation. 

The result is that the Program for Educationel Development is quite unlike most other 
education or research programmes. Involvement is voluntary for all but a few support staff. 
It attracts a wide range of individuals with some interest in educational research and 
development. The tangible rewards for those who are involved are an occasional confe- 
rence or publication. However, we hope that these are matched by the intangible rewards 
which come from the opportunity to share ideas about education and engage in critical 
analysis and research of educational problems. Presently about 40 academics are affiliated 
with PED. They represent most departments in the Faculty of Health Sciences. Much of 
the work is done together by these individuals and support staff, some five research 
associates, two administrative staff, and three secretaries. Most of our financial support is 
derived from outsit sources, and the remainder from University funds. For many years 
the external funding came from small research grants, but more recently we have been 
fortunate to attract more long term funding for several large research projects. 

In contrast to many offices of medical education, we have a less direct accountability to 
the educational programmes. The philosophy of the Faculty has always been that the 
educational programmes are owned by all faculty members, not by an educational elite. 
The idea of placing large management responsibility for the educational programmes of 
the Faculty in the hands of a few professional educators is not consistent with the Faculty' s 
philosophical stance. 
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There are, of course, some service responsibilities in any school that are necessary for the 
survival of educational programmes. Faculty development (e.g. tutor training workshops, 
problem-based learning workshops, leadership programmes) was originally developed by 
some individuals in PED, it then became a separate arm of PED, and it has now become 
an independent programme. Learning resources development has similarly evolved from 
an activity of the Audio-visual Department to a separate office in the Faculty. A joint 
executive was created to manage the overlap among the three areas (faculty development, 
learning resources, research), but perhaps the degree of shared activity is too little. For 
example, there has beenlittleefforttorigorously evaluate the impact of faculty development 
activities or to explore alternative models of learning resource design. 

What, then, is the major role of PED within the Faculty? Its members act in the role of 
consultant and facilitator. Our mission is, in part, to keep ourselves informed of the 
literature and other developments in education outside McMaster, so that our educational 
leaders can make informed and rational decisions. We are also available to offer advice and 
support for initiatives in education. These range from studies of innovations, like the 
introduction of OSCE assessments into the nursing programme or study of the reliability 
of the admission's interview in the occupational arid physiotherapy programmes, to 
ongoing support of major innovations like the major curriculum changes in the M.D. 
programme in the early 1980's. In part, this is achieved by the Faculty's financial 
contribution to PED - one research associate is paid by the Faculty to support ongoing 
educational developments. In addition, academics and research staff, whose primary 
responsibility is to specific externally funded projects, may contribute some of their time 
and expertise to internal projects. 

PED has had a long standing interest in stimulating educational research through faculty 
development. We have sponsored a Fellowship programme for many years. This attracts 
from one to four fellows each year from around the world for durations of three months to 
a full year. Fellows can base themselves in PED and essentially design their own 
programme, ranging from the very research-orientated, with attachment to a research 
group, or courses in design and statistics, to a very much more practical orientation, with 
workshops in tutor training, problem-based learning, and co-tutoring. There is no tuition 
or other fee, and we can provide some facilities and support (e.g. computers, limited 
secretarial support) for fellows. More recently, we joined with the University of Toronto 
in their M.Sc. in Medical Education, offered through the Ontario Institute for Studies in 
Education. This will make it easier for our own faculty staff to pursue advanced degrees 
in education. 

We also use a number of other means to stimulate interest and involvement in educational 
issues. Education Rounds are held once a month, on Wednesday at noon. About two years 
ago, we developed a "Discussion Forum in Education" to give educational leaders the 
opportunity to discuss educational issues of local concern and to stimulate research on the 
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issues. PED members also meet at Education Research Seminars on Friday morning once 
a month to hear about work in progress. 

Advantages and Disadvantages 

The organisation of PED is in some respects a microcosm of the organisation of the 
Faculty. This involves an interplay between departments and educational programmes. At 
McMaster, departments do not quite resemble feudal fief doms to the extent they do in 
many other Faculties; and PED, with its base on the programme axis of the matrix, can 
move freely within the departmental structure. This framework has permitted maximum 
flexibility. However, the Negative side is some sacrifice of visibility and coherence, so that 
often no one seems to really know what is happening. Individual faculty members, even 
those who are card carrying members of PED, frequently have trouble in identifying with 
it as a corporate whole. 

The structure has some disadvantages: it is difficult to recruit faculty members specifically 
for their expertise in educational problems, as priorities for recruitment are established, by 
and large, by the chairmen of departments. Yet, it is difficult to envision any alternative 
arrangement which would provide the flexibility that PED enjoys. It has enabled the 
programme to be responsive to the majority of educational programmes in the Faculty, and 
it has permitted us to involve academics from nearly all departments. 

PED's heavy dependence on external funding has been a double-edged sword. It has 
clearly kept us "lean and mean" through the years, and it has contributed immeasurably to 
our high productivity in medical education research. It has also permitted us to be 
maximally responsive to short term demands, as we have used our resources to highlight 
educational issues which involve a cross-section of faculty members in educational 
research. This creates a large cadre of individuals to address internal as well as external 
demands and priorities. A disadvantage is that the major unit of activity is often the 
research project, and many members, both academics and staff, quite appropriately, see 
their attachment to the principal investigator and the project first, and to PED second. 
Reliance on external funds has also contributed to an anxiety among support staff with 
regard to future funding arrangements. At present, funding is stable, but this period will 
inevitably come to an end. The arrangement presents a constant challenge to the leadership 
to devise ways to encourage all to see themselves as part of a larger corporate enterprise, 
and also to ensure that they are valued beyond the immediate horizon of their present 
research grant. 

It is clear that we have traded off the investment in a cadre of individuals with great 
expertise in education, for a much larger aiTay of faculty members, with individually 
considerably less expertise. We have relied on our ability to nurture individuals so that they 
can, in time, emerge as competent first class members of the educational research/ 
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dev .«opment community. There are a few notable successes, but at times the process 
appears to proceed at a snail's pace. 

Nevertheless, on balance, we believe the advantages outweigh the disadvantages. George 
Miller (198 1) has been very complimentary of this model: "... it was Steiger*s view in 197 1 
that the most impressive integration of educational science into the work of a medical 
school was to be found not in any of those with large educational centers, but at McMaster, 
which had no formal educational research and development unit. It still has only a 
"Program for Educational Development..." There have been several occasions over the 
years when opportunity existed to change ourselves into the model of a large educational 
centre. Each time, on careful reflection, we decided we liked ourselves "just the way we 
are". Current research and development activities are summarised in the Appendix. 

Reference 

Miller, G.E. ( J 981). Educating medical teachers. Cambridge, MA: Harvard University 
Press. 
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Appendix 

Current Research and Development Activities 
Professional Behaviours (Bienenstock, Keane, Eaman) 

This group has been active in defining and measuring "professional behaviours" of 
medical students, which can be observed and assessed in the small group setting and which 
are predictive of behaviours used with patients in the clinical setting. To date, considerable 
effort has gone into developing scales, and studies of reliability of observations in the small 
group and in the clinical setting are now underway. 

Evaluation of Quality of Teaching (Wakefield, Keane) 

This group of individuals in both Faculty Development and PED has been attempting to 
derive methods for the evaluation of quality of teaching. A literature review was 
conducted*, and a faculty survey of attitudes towards teaching and evaluation of teaching 
has been completed. The results are being analysed and will be used to suggest future 
directions. 

* See paper by Wakefield and Keane in this Volume of the Annals. 
Clinical Reasoning (Norman, Brooks) 

This long-standing interest group has recently undertaken studies in visual domains such 
as dermatology and radiology. The major goal is to understand the organisation of 
knowledge and its development with expertise. The studies to date have provided an 
explanation for observer variation in some clinical domains, and have implications for the 
teaching of methods of clinical diagnosis. 

Physician Review Program (PREP) (Davis, Premi, Hannah, Norman) 
This group was formed three years ago in response to a request from the College of 
Physicians and Surgeons of Ontario. The group has developed methods for the educational 
diagnosis of physicians referred by the College with concerns about competence. A 
number of studies supporting the reliability and validity of the multidimensional approach 
to evaluation has been conducted. 

Geriatric Education (Blumberg, Ryan, Deveau) 

A number of activities in geriatric education are being conducted by the Geriatric 
Educational Development Unit, a component of the Educational Centre for Aging and 
Health. They have reviewed and revised many health care problems used in the undergraduate 
education programmes to include relevant and accurate data related to aging. Current 
research is invest ; gating the effect of patient age in health care problems on student 
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learning. As this unit is charged with evaluating the entire Educational Centre for Aging 
and Health, they are collecting quantitative and qualitative data from programmes. 
Educating Future Physicians of Ontario (Neufeld, Shannon, Pickering, Norman) 
This major activity involving all five medical schools in Ontario consists of three 
components. Component I is developing methods to assess the health care needs of Ontario 
by interviewing consumer groups, practitioners, etc. Component II is developing faculty 
development programmes applicable to the medical schools. Component III will develop 
assessment methods to assess the students 9 mastery of these skills. Specific methods have 
been developed to assess ethical skills, critical appraisal skills, and interpersonal skills in 
an objective, structured clinical examination (OSCE) format. The administrative operation 
of Component I and some operations of Component III are housed at McMaster. 

Studies in Radiology (Norman, Coblentz) 

This group is exploring some of the determinants of observer variation in radiology. 
Studies of the diagnosis of bronchiolitis have been completed and new studies of 
mammography arc underway. 

Studies of the Careers of McMaster Graduates (Woodward, McAuley, Ferrier) 
A series of studies documenting the outcomes of medical education at McMaster has been 
undertaken, using surveys of graduates and postgraduate supervisors, secondary data 
sources have been used to explore similarities and differences in the practice performance 
patterns of graduates from McMaster and other Ontario schools.* 

* See paper by Woodward et al. in this Volume of the Annals. 

Studies of the Impact of Women on Health Care Delivery (Woodward, Cohen, Ferrier) 
These investigations are exploring how career development of men and women physicians 
are similar and different, and the factors underlying the differences. 

Studies of aspects of the M.D. Program (Blumberg, Cohen) 

These studies address several important aspects of the process of education in the 
undergraduate medical programme. One study examined students who were given 
unsatisfactory assessments to determine when these occurred, and if the students 9 
difficulties could have been predicted earlier from review of the written reports on 
students. Another study surveyed education coordinators at nine medical schools to 
determine attitude toward "forward-feeding" of assessment information about students. 
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MEDICAL EDUCATION WORLDWIDE 
World Federation for Medical Education 

Henry Walton 

Edinburgh, United Kingdom 

The main activities of the Central Office of the World Federation for Medical Education 
(WFME), can be summarised as follows: 

1 . Reorientatic a of medical education at the global level is being promoted. Collaboration 
with the World Health Organisation Headquarters occurs intensively. In addition, close 
participation in the work of UNICEF is regarded as a priority, and active cooperation in 
the medical education field with UNESCO advances. 

2. Support is given to regional activity in the medical education and health care sectors, 
through joint activity with the six Regional Offices of the World Health Organisation and 
with the six Regional Associations for Medical Education of WFME. The Ministerial 
Consultation in Africa (Ministerial Consultation for Medical Education in Africa, 1989), 
for example, with the WHO Regional Office for Africa and the Nigerian Government 
(Professor Ransome-Kuti), resulted in major developments in Africa (the Task Force) and 
Intensive visits to African countries by the Secretary of Health of the U.S., Dr. Louis 
Sullivan (Sullivan, 1991). 

3. Approaches from national governments have been responded to, in particular ft\>m 
ministers of health and their advisers. An example was the dialogue between WFME and 
Iraq during the Gulf War and subsequently, and now with Yugoslavia. The Federation 
meets with Health Ministers, as in the former USSR recently, and makes the policies of 
ministers widely known. 

4. National associations for medical education, either through their regional associations 
or directly to the World Federation, have been assisted whenever possible. Such assistance 
is critical in India, for example, and in African countries. 

5. The projects, specified at the 1988 World Conference and subsequently (World 
Federation for Medical Education, 1 988) , as a major aspect of the International Collaborative 
Programme for Reorientation of Medical Education, have been initiated, at global, 

Requests for reprints should be sent to Henry J. Walton, President World Federation for Medical Education, 
University of Edinburgh, Centre for Medical Education, 11 Hill Square, Edinburgh EH8 9DR, United 
Kingdom. 
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regional, national and institutional levels. In Europe, for example, medical education in 
Portugal in all its stages is being reoriented, by WFME, UNESCO, WHO BURO and the 
Portuguese Government. A major regional Continuing Medical Education project, with 
WHO BURO, involves all countries of Europe. 



Development Plan for WFME Central Office 

The World Federation for Medical Education, the University of Edinburgh, the Centre for 
Medical Education of its Faculty of Medicine, the City of Edinburgh District Council and 
the Scottish Development Agency completed the Development Plan (World Federation 
for Medical Education, 1991) ensuring a longterm Central Office as a base in Edinburgh 
for the World Federation for Medical Education, to implement the International Collaborative 
Programme for Reorientation of Medical Education as mandated at the World Conference 
in 1988. 

The new address of the World Federation for Medical Education is: The University of 
Edinburgh Centre for Medical Education, 1 1 Hill Square, Edinburgh, EH8 9DR, United 
Kingdom. 



The Executive Council of WFME 

The Executive Council will meet in Edinburgh on 14-15 April 1992: Professor Henry 
Walton, Chairman; Dr. Pablo Pulido, Executive Director, Panamerican Federation of 
Associations of Medical Schools; Dr. Jasbir Bajaj, President, South East Asian Regional 
Association for Medical Education,; Dr. Mahmud Mohd Nor, President, Western Pacific 
Regional Association for Medical Education; Professor Albert Oriol-Bosch, President, 
Association for Medical Education in Europe; Dr. Bashir Hamad, P: isident, Association 
for Medical Education in the Eastern Mediterranean Region; Professcr Kopano Mukelabai, 
President, Association of Medical Schools in Africa; Dr. Andre Wynen, World Medical 
Association, Brussels; and Dr. Eric Goon and Dr. Charles Boelen, Division of Human 
Resources for Health, WHO Geneva. 

Interna 

tional Collaborative Programme 

Progress with four of the projects will be mentioned: 

1. Statement on Medical Education in Europe 

Under the auspices of WFME, an enquiry was carried out and recommendations in nine 
key areas of the training of doctors were agreed at Heraklion, Crete on 26 September 1 989 
(Mediterranean Medical Society, 1990) , with the full participation of the WHO European 
Office. The Association of Medical Deans in Europe has endorsed the Statement, which 
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has been extensively reviewed by all medical schools in the Region. 

2. The Portuguese Project 

The Portuguese Project, to reorientate medical education in Portugal as decided at the 
Lisbon Ministerial Consultation in Europe, has completed its first stage. The guiding body 
is the Interrnimsterial Commission in Portugal, assisted by an International Committee 
which Professor Walton chairs, and on which the European Office of WHO is fully 
represented. UNESCO sponsors the project with the Government of Portugal. 

3. The CME in Europe Project 

The steering Group has set up a Task Force with representatives nominated from all 
countries in Europe. This is the opening phase of a planned extension to a global project. 

4. Global Curricula 

WFME is developing curricula in six specialties with the corresponding international 
specialist Federations: Neurology, Public Health, Paediatrics, Otorhinolaryngology , General 
Practice ana Child Psychiatry. 



Collaboration with the World Health Organisation, UNESCO and 
UNICEF 

WFME works in close collaboration with the United Nations and its specialised agencies, 
especially WHO, UNICEF and UNESCO. 



World Health Organization 

WFME is represented as a non-governmental member in official relations, at the meetings 
of the Advisory Board of WHO in Geneva, at the World Health Assembly in Geneva, and 
at the meetings of the Regional Committees annually of the six WHO Regional Offices. 
WFME works particularly closely with the Regional Directors and with the Division for 
Development of Human Resources for Health (HRH) in the Geneva Office. 
WFME is actively associated in the implementation of the important HRH document, 
"Changing Medical Education: An Agenda for Action" (World Health Organization, 
1991). 



United Nations Children's Fund 

WFME is in close association with UNICEF, the main links with Dr. James P. Grant, Dr. 
Nyi Ny i, Dr. Ranjit Atapattu and Ms. Mary Cahill. The Federation on all possible occasions 
seeks to promote UNICEFs aims. The notable UNICEF World Summit for Children 
which took place in New York in September 1990, has resulted in resolutions many of 
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which are relevant to medical education and the training of physicians. 



United Nations Educational Scientific and Cultural Organisation 
A contract has been signed by UNESCO with WFME, in support of the Portuguese Project. 
The Director General, Dr. Federico Mayor, is represented by Dr. Claude Rosenfeld on the 
International Committee. 

The Network of Community-oriented Educational Institutions in the 
Health Sciences 

WFME seeks the closest possible partnership with the Network. 
Promotion of the Edinburgh Declaration 

The endorsement of the Declaration has been extraordinarily widespread3* and it is now 
extremely well known and reference to it is usual when reform of die training of doctors 
is under discussion. With the help of WHO Geneva and the six WHO Regional Offices, 
the 12 principles of the Declaration will be more widely conveyed to as many teachers or 
doctors as possible. 

Implementation of the International Collaborative Programme for Reorientation of Medical 
Education continues, by development of further global, regional, national and institutional 
projects for reorienting the training of doctors as detailed in the Report of the World 
Conference on Medical Education, and the Reports of the Ministerial Consultations when 
Ministers of Health and Ministers of Education were brought together by WFME, the 
relevant WHO Regional Office, and a host Government, e.g. Nigeria in the case of Africa, 
and Portugal for Europe. 

(see "Ministerial Consultation for Medical Education in Europe: Report of a WHO 
Meeting", EUR/ICP/HMD/1 15, and "Medical Education in Africa: An Agenda for 
Change", Ibadan 1989) 



Implementation of the WHO HFA Policy 

As has been stated frequendy at the World Health Assembly, the Edinburgh Declaration 
in the field of health professions education has the standing the Alma Ata Declaration has 
in the field of health care. The WHO and the WFME nolicy run in parallel, and as far as 
the latter body is concerned will continue to do so. 
Dr. Louis Sullivan has stated2 : 

"I have followed very carefully the work of the World Federation for Medical Education, 
the international organisation concerned with medical training based at the University of 
Edinburgh, The Federation is attempting, in the words of President Henry Walton, to 
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implement a Programme of wide reassessment of the training of doctors on a global level, 
and to reorient medical education to be as relevant as possible to contemporary health 
needs of communities". 

This WFME policy will coninue actively, at the level of action now that a mandate for 
reform exists. 



Second World Conference on Medical Education 

It has been proposed widely that a further World Conference on Medical Education should 
take place in 1993. Planning Groups will meet meanwhile, and the proposal is that the six 
Regional Associations should consider the World Conference their major Regional 
activity, WHO has agreed in principal to be a main co-sponsor, and the full co-sponsorship 
of the six WHO Regional Offices hau been given by the six Regional Directors. 



Fifth International Conference on Assessment of Clinical Competence 
This conference, co-sponsored by WFME. will be held at the Centre for Medical 
Education, Dundee, on 1-3 September 1992. 



The Mediterranean Medical Society 

The Society is potentially of great importance in improving health care. It operates in, and 
therefore brings together, the spheres of four WHO Offices: the Geneva Headquarters, 
AFRO, EMRO and EURO. The Society has two main aims: the improvement of health 
care, and reform of medical education and the training of doctors. The Second Meeting of 
the Society will be held in Athens on 2 1 -25 June 1992, and will be addressed by the WFME 
President and Dr. Charles Boelen, HRH Division, WHO Geneva. 

Prior 

ities decided by the WHO Regional Directors 

In promoting the WFME International Collaborative Programme, each Regional Director 
has designated a priority area: 

I. Dr. L.G. Monekosso, AFRO: Community Based Training 

II. Dr. Carlyle Guerra de Macedo, PAHO: 

a) Fostering the access to scientific information. 

b) Medici education, pattern of practice and health needs. 

III. Dr. H.A. C :?&iry, EMRO: Community Oriented and Community Based Medical 
EducaSirr: 

IV. Dr. U. Ko Ko, SEARO: Practicalities of reorienting medical education. 
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V. Dr. ST. Han, WPRO: Basic Medical Training. 

VI. Dr. J.E. Asvall, EURO: Continuing Medical Education. 



Summary 

All the aims of WFME stand to be augmented by the partnership with the Network, which 
the two bodies have recently reconfirmed. 
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Changing Medical Education: An Agenda for Action 

Charles Boelen 
Geneva, Switzerland 

The physician of tomorrow should be able to respond better to the needs of communities. 
Therefore, on the one hand, he will need to possess the competences necessary to promote 
healthy lifestyles and to communicate with consumers and community leaders in order to 
obtain their involvement in health activities. On the other hand, he should be capable of 
critically applying the latest technologies in health sciences and making decisions that take 
into account ethical, financial and continuation of care issues, striking a balance between 
the expectations of patients and of society at large. In order to have a better understanding 
of health needs in relation to the total requirements of an individual and of the community, 
he should be capable of continuing his work with others even outside the health sector, and 
of making better use of information and managerial techniques in planning and performing 
his duties. 

Dramatic changes will be required in medical practice; these will call for important 
interventions, among which is an equally dramatic change in medical education. Changes 
in basic medical education will actually affect the delivery of health services only 10 to 1 5 
years later. Therefore, national authorities, training institutions, and professional associations 
are urgently called upon to initate and support a movement that will strive to educate the 
new medical and health work force and re-educate the existing one to respond to society 9 s 
emerging needs and demands. 

It is higher education that beares the responsibility for preparing a manpower in line with 
a prospective vision of what society ' s future needs and demands will be 1 . As for the medical 
profession, not only should it anticipate the nature of the education required, but is should 
also contribute to finding appropriate ways to optimise the use of human resources for 
health. Medical schools and other university bodies should and can take measures to use 
their potential and resources accordingly. In the absence of such initiatives., forces outside 
the academic and professional world may well take the lead and impose changes that may 
not fully involve medical education and the medical profession. 

An understanding must be reached on what is meant by "quality medical education" in 
order to design and apply meaningful strategies to achieve it and to know what to monitor. 

Copies of the full document, with its more specific proposals, may be obtained from Charles Boelen, Chief 
Medical Officer for Educational Development of Human Resources for Health, World Health Organization, 
121 1 Geneva 27, Switzerland. 
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The assumption is that, if countries or institutions were able to determine as objectively as 
possible to what extent advocated changes apply to them, they would be more inclined to 
accept the idea of reorienting medical education. These values entailed in "quality medical 
education" would influence the setting of standards in medical education and would serve 
as goals towards which strategies for change would be aimed. Indicators and criterial 
would be proposed to assess, in terms as quantifiable as possible, to what extent current 
situations meet the standards that have been decided upon. 

Because of the complexity of the change process in medical education and because of a 
variety of determinants that may influence it, depending on the peculiarities of the political 
and sociocultural context., there is no unique treatment for all possible situations. 
Itineraries leading to the goal of the proposed new mandate may vary form place to place, 
but the goal should essentially be the same. Several strategic approaches should be 
proposed as optional entry points into the voyage towards change. The formulation and 
implementation of strategies for change are to be considered as action research projects. 
Protocols would be developed outlining the main characteristics of each strategy, and then 
proposed to the main actors in the reform of medical education so as to make them fully 
aware of the requirements and constraints entailed. 

A close watch should be maintained on worthwhile endeavours and achievements in 
adjusting medical education that result in the betterment of people's health. Countries, 
institutions, and individuals engaged in a change process are always eager to learn from 
the experiences of others so as to improve their own practices. Keeping each other up to 
date creates solidarity among those who have the courage and take the risk to break with 
the status-quo, and this encourages them to go further down the road. 
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Research in Medical and Health Professional Education: Current 
Bibliographyt 

Kenneth V. Jones and John Noyce 

In late 1 989, the ANZ AME* Executive devoted the major part of one of its meetings to the 
consideration of ways to encourage research in medical and health professional education. 
Many problems hindering medical education research were discussed - lack of targeted 
funding; lack of support, and even interest, within institutions; and the uncertainties of 
would-be researchers with regard to the literature in this area. It was in response to this last 
issue that this project developed. The aim of the project was to identify some of these 
uncertainties, and to examine ways in which they might be dealt with. The central feature 
of this project was the development, by John Noyce, of a Current Bibliography of Journals 
- supported by an ANZAME Seeding Grant in Medical Education - which accompanies 
this paper. This paper considers the question of how to utilise the literature simply and 
economically. 

Content Experts or Educators? 

Most ot the problems of would-be researchers in the field of medical education (and even 
of consumers of this research) are the result of an institutional reality: the vast majority of 
us were hired for our expertise in a clinical or scientific content area, not for our background 
in, or even our capability for, education. The tertiary sector remains the only education 
sector where teachers are not required to be trained as teachers. The assumptions that "good 
research will produce good teaching", and that University students are sufficiently 
motivated and mature that a lot of teaching technique is unnecessary", are still common, 
especially within the universities. With the latest round of institutional amalgamations in 
Australia, many more health professional educators are now in the universities. This has 
meant a greater exposure to the "publish or perish" ethos, and, regrettably, sometimes even 
a devaluing of teaching. Educators* time is limited, and the necessity to keep up with the 
literature in one* s content area constrains one's ability to keep up in another. Even though 
dissatisfaction with existing courses and motivation for improvement may be high within 
our schools and faculties, uncertainty about tackling the educational literature may inhibit 
us from being systematic about change, and even from attempting change. 

* Australian and New Zealand Association for Medical Education. 

t Reproduced by kind permission from Jones, K.V., & Noyce, J. (1991). Research in 
medical and health professional education: Current bibliography. ANZAME Bulletin, 1 8, 
13-25. 

Address for correspondence: K.V. Jones, Department of Psychological Medicine, Monash Medical Centre, 
Prince Henry's Hospital Melbourne, Victoria 3004, Australia. 
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As Consumers of Information 

The usual motivation to seek information is a sense that things are not going well, or that 
there could be a better way to do things. Few of us will have had the experience of finding 
a perfect solution, and wanting to check to see if anyone else knows about it yet In any case, 
since we are primarily educators rather than educationists, change tends to underlie our 
search for information. 

In order to produce change from within the educational institutions, it is necessary to have 
a clear idea of what needs changing, and what can be changed. We are all aware of areas 
in our own courses that the students find boring, or of questionable value. The decision to 
change these elements needs to be based on data, however, because there may well be 
interest and value in those areas that are simply being hidden by poor educational 
approaches. 

Alternatively, change may be forced from outside the institutions. Community pressure, 
influence of professional bodies and institutional constraints may all force changes in 
courses - often without at the same time providing further direction to the change 
demanded. The result can be disaster, unless we know what we arc doing. In circumstances 
such as these, we need to become informed consumers of the literature in medical 
education. The questions to be considered include: 1) What is known about the approach 
now in use? What works and what doesn't? How do we keep the baby while disposing of 
the bathwater? Can sufficient improvement be achieved within the present model (for 
example, through development of teaching skills in our teachers), or is an entirely different 
model needed? 2) What alternative approaches exist? How have others dealt with the same 
kind of problems? What has worked because of its intrinsic appropriateness to one setting, 
and what has worked because of common principles? (This is a particularly complex 
question v/hen the best answers may come from disciplines other than our own. 3) How 
ought changes to be prepared, "sold" to colleagues, and/or implemented? What mistakes 
have resulted in the failure of desirable changes? How can stumbling blocks be overcome? 

As Transmitters of Information 

On the other hand, we may have obtained information that we wish to share with others. 
This could range from information that has been acquired incidentally - as part of 
evaluation of a course, for example, or from experience as a teacher - to carefully planned 
research results obtained specifically to test an hypothesis. As potential transmitters of 
information, particularly as first time transmitters, a different set of uncertainties about the 
literature may affect us. 

1 . Is the information "old hat"? Have others been doing this for so many years that they do 
not even think it worthy to comment on? The most orthodox use of the literature may be 
to make sure that old mistakes are not repeated, and that we are not constantly "re-inventing 
the wheel". This sounds simple until one realises that Excerpta Medica alone covers some 
4,700 journals. 



319 



2. Is the information naive? How does it fit into educational thinking? Is there a better way 
to conceptualise it? Is some unrecognised element at work, different from the one that we 
believe to be critical? 

3. Is there anyone out there who wants to hear about these results? If so, who? How do I 
reach them? What is the most direct path? Sh<^ '!d a broad or narrow range of consumers 
be the target? 

Reducing Uncertainty 

Whether as a consumer or a transmitter of information, there are a few rules that may help 
in the reduction of uncertainty about the medical education literature. 

1 . The task is not as big as it may appear. You will note from the bibliography that the 
specialist medical and health professional education journals all fit on one page. The big 
problem is locating these journals as not all may be in the library at a give) i institution. Once 
located, however, a brief session in the library from time to time will enable anyone to keep 
up with what's in these, and whether any of it is relevant. This may iead you to specific 
references in other journals that should be read. 

2. The task can be reduced. Admittedly, the number of journals that occasionally print 
education articles is large. However, it is usually possible to decide on a fraction of it that 
is most likely to be useful. If the question is specific to one profession, or a subspecialty, 
only the literature for that profession or subspecialty is urgent. If it is concerned with broad 
educational principle, this specialist literature can be set aside, and the educational 
literature becomes the focus of attention. 

3. There are shortcuts. The shortest may be to ask somebody. In each institution there will 
be several people who share an interest in educational issues. By talking to one another, 
the task of finding out what's going on, and where it has been written up, may be reduced. 
Experts in the area are usually happy to help out non-experts. 

Tackling the Literature 
L Journals 

The appended list of journals dealing with medical education involved much hard work; 
we hope you find it useful. To find Australian library locations of a specific journal you 
will need to consult the microfiche National Union Catalogue of Serials available in most 
libraries. You may need to use the earlier Scientific Serials In Australian Libraries for older 
titles. 

For subject listings of journals in Medicine use List Of Journals Indexed In Index Medicus t 
and also Ulrich's International Periodical Directory. 

2. Databases 

The maj or database in the medical field is MEDLINE produced by the US National Library 
of Medicine. It is available online and, mote recently, in CD-ROM format. Its print 
counterpart is INDEX MEDICUS (Medline also includes indexing from Index to Dental 
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Literature and International Nursing Index). The CD-ROM version is available through 
eight different publishers, with the Silver Platter version the most popular in Australia 
(distributed by Aldus). To use it properly you will need an initial session with an 
experienced user (your librarian will be pleased to assist), and some familiarity with 
MeSH- the Medical Subject Headings Thesaurus (Medical Education is under "Education, 
Medical" but Nursing Education is under "Nursing..."). For a quick search of recent years 
the MEDLARS CD-ROMs are invaluable, but for a comprehensive search (1966 to date), 
you will still need an outline MEDLARS search which should be done by an experienced 
librarian. It is important also to realise when using the CD-ROMs that only the major 
Australian and New Zealand medical titles are indexed on MEDLARS. For the rest of the 
ANZ titles there is a special subset on the online Australian Medline Network (Medlars can 
also be searched through a variety of other online hosts). 

The great rival to Index Medicus is EXCER?TA MEDICA which is issued in 52 printed 
sections, each covering a major subject, e.g., cardiology, psychiatry. The journals 
coverage is greater (4,700 to 2,700 on Medline/Index Medicus). The CD-ROM versi ons arc 
now becoming available in Australia on Silver Platter (Aldus). 

Other databases which you may need to use, depending on subject, include: 

BIOLOGICAL ABSTRACTS (printed, online & CD-ROM) 

PSYCHUT (the online/CD-ROM version of the printed Psychological Abstracts) 

ERIC (education: print and CD-ROM version) 

UFE SCIENCES COLLECTION (CD-ROM) 

CUNMED CD (CD-ROM) 

HEALTHPLAN-CD (CD-ROM) 

CINAHL (CD-ROM version of the printed Cumulative Index to Nursing and Allied Health 
literature) 

(see R. Farbey, Medical Databases 1988 (3rd ed.), London: Aslib, 1987; and R.K. Fryer 
et al. Beyond Medline: a review of ten non-Medline CD-ROM databases for the health 
sciences, Laserdisk Professional, May 1989: 27-39). 

For scanning current issues of medical journals, CURRENT CONTENTS: UFE SCIENCES 
and CURRENT CONTENTS: CUNICAL MEDICINE arc invaluable. Both arc now also 
available on floppy disc from the US Institute for Scientific Information (ISI). 

If you have details of a key paper, author, or organisation in your subject, use can be made 
of another ISI publication, the SCIENCE CITATION INDEX. When using the print version 
you will need a magnifying glass and much patience. There is a CD-ROM version. 

3. Monographs 

Good general books on education can serve as an excellent entry point to the literature. 
They range widely where journal articles may only provide a narrow focus. In fact, a well 



30.6 



321 



written monograph can be very similar to consulting a friendly expert. Some examples that 
may be useful include: 

Newble, D., & Cannon, R. (1987). A handbook for medical teachers. 2nd ed. Lancaster, 

UK: MTP Press, 157 p. 
This well-presented handbook from two authors at the University of Adelaide is a good 
introduction to the field of medical education. 

Other titles of interest include: 

Barrows, H.S., & Tamblyn, R.M. (1980). Problem-based learning: An approach to medical 

education. New York, NY: Springer Publishing Co. 
Cox, K.R., & Ewan, C.E. (Eds.) (1987). The medical teacher. 2nd ed. Edinburgh, United 

Kingdom: Churchill-Livingstone. 
Hubbard, J.P. (1971). Measuring medical education. Philadelphia, PA: Lea & Febiger. 
Miller, G., & Abrahamson, S. (1961). Teaching and learning in medical school. Harvard: 

Harvard University Press. 

For health professions: 

Boissoneau, R. (1980). Continuing education in the health professions. Rockville, MA: 
Aspen. 

Ford, C.W., & Morgan, M.K. (Eds.). (1976). Teaching in the health professions. St. Louis, 
MI:Mosby. - 

McGuire, C.H. et al. (1983). Handbook of health professions education. San Francisco, 
CA: Jossey-Bass. 

Several organisations produce monographic series, notably the (UK) Association for the 
Study of Medical Education (ASME) whose Booklets are also printed in the journal, 
Medical Education. 

Several medical education centres produce regular monographs: 

- Centre for Medical Education, University of Dundee, Scotland. 

- Division of Research in Medical Education, University of Southern California, Los 
Angeles. 

- Centre for Educational Development, University of Illinois College of Medicine, 
Chicago. 

- School of Medical Education (formerly Centre for Medical Education, Research and 
Development), University of New South Wales, Australia. 

4. Conferences 

The Association of American Medical Colleges publishes the Proceedings of \hc RESEARCH 
IN MEDICAL EDUCATION CONFERENCE, which is the major annual gathering for 
medical educators in North America. 
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The Josiah Macy Jr. Foundation (US A) publishes the proceedings of its regular seminars 
and conferences on medical education (the 1988 seminar was on Clinical Education). 

The Proceedings of the WORLD CONFERENCE ON MEDICAL EDUCATION are 
published by the World Federation for Medical Education. 

5. Theses and Dissertations 

DISSERTATION ABSTRACTS INTERNATIONAL lists doctoral theses with abstracts from 
North American universities. It is available online, in print, and on CD-ROM. For the 
United Kingdom see the ASUB INDEX TO THESES. For dental theses see INDEX TO 
DENTAL LITERATURE. For nursing theses see INTERNATIONAL NURSING INDEX. 

Continuing the Process 

We hope the following bibliography will prove useful to you. In each case the information 
provided is title, [source], frequency of publication, and a recent example of relevance to . 
medical and/or health professional education. It is divided into sections as follows: 

1 . Specialist medical and health professional journals 

2. Higher education journals containing articles of relevance to medical and health 
professional educators 

3 a. General medical journals containing occasional education articles 

b. Sub-specialist medical journals containing occasional education articles 

4 a. General nursing journals containing occasional education articles 

b. Sub-specialist nursing journals containing occasional education articles 
5. Health specialist journals, excluding medical and nursing, containing occasional 
education articles. 

If there are other journals you find useful, please let us know, as we hope to keep this list 
current and distribute it widely. 

We wish to thank Neil Paget for his comments on a draft of this paper. 
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CURRENT BIBLIOGRAPHY OF JOURNALS 

John Noyce and Kenneth V. Jones 

1. Specialist Medical and Health Professional Education Journals 

ACADEMIC MEDICINE 

[Association of American Medical Colleges (Formerly Journal of Medical 

Education)] 12pa. 
eg. Medical school admission (65(3) Mar 1990: 1 33-1 89) 

ANZAME BULLETIN 

[Australasian and New Zealand Association for Medical Education] 4pa 
eg. Assessing student performance in the clinical setting (16(2) Apr 1989: 8-17) 

JOURNAL OF CONTINUING EDUCATION IN NURSING 
[USA] 6pa 

eg. Communication skills for continuing education in :iursing (21(3) May 1990: 109- 
112) 

JOURNAL OF NURSING EDUCATION 
[USA] 9pa 

eg. Nursing students' perceptions of anxiety-producing situations in the clinical setting 
(29(4) Apr 1990: 183-187) 

MEDICAL EDUCATION 

[Association for the Study of Medical Education (UK)] 6pa 
eg. Medical student experience of London general practice teaching attachments (24(4) 

July 1990: 354-358) 

MEDICAL TEACHER: AN INTERNATIONAL JOURNAL OF MEDICAL 
EDUCATION 
[UK]4pa 

eg. Medical ethics education in the eighties: a study of five medical schools ( 1 1 (2) 1 989: 
177-185) 

NURSE EDUCATION TODAY 
[UK]6pa 

eg. The lecture as a teaching method in modem nurse education ( 1 0(4) Aug 1 990: 290- 
293) 
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NURSE EDUCATOR 
[USA] 6pa 

eg. Cheating among nursing students (15(2) May 1990: 32-35) 

REVIEW OF ALLIED HEALTH EDUCATION 

[USA] irregular (4: 1981 ; 5: 1985) 
eg. Accreditation in the allied health professions (5, 1985: 107-125) 

TEACHING AND LEARNING IN MEDICINE: AN INTERNATIONAL JOURNAL 
[USA] 4pa 

eg. The right place at the right time: an example of innovation in medical education 
(New Zealand) (1(3)1989: 171-175) 

UNIVERSITY OF NEW SOUTH WALES SCHOOL OF MEDICAL EDUCATION 
NEWSLETTER 

[Australia] 2pa 

2. Higher Education Journals Containing Articles of Relevance to 
Medical and other Health Professional Educators 

ASSESSMENT AND EVALUATION IN HIGHER EDUCATION: AN INTERNA- 
TIONAL JOURNAL 

[University of Bath (UK)] 3pa 
eg. Evaluation of a course in interviewing for first year medical students ( 1 2(3) Autumn 

1987: 191-201) 

AUSTRALIAN EDUCATION REVIEW 

[Australian Council for Education Research] 2pa 
eg. The profession of occupational therapy (30, 1990: 97-108) 

AUSTRALIAN JOURNAL OF ADULT AND COMMUNITY EDUCATION 
[Australian Association of Adult and Community Education] 3pa 

eg. Continuing education for the health sciences (especially occupational therapy) 
(27(2) 1987: 22-28) 

AUSTRALIAN SCIENCE TEACHERS JOURNAL 

[Australian Science Trachers' Association] 4pa 
eg. Some important areas of student assessment in science teaching (35(4) Nov 1989: 

71-79) 
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BRITISH JOURNAL OF EDUCATIONAL PSYCHOLOGY 

[British Psychological Society] 3pa 
eg. Modelling medical decision-making under uncertainty (59(2) June 1989: 1 87-199) 

BRITISH JOURNAL OF EDUCATIONAL TECHNOLOGY 

[National Council for Educational Technology] 3pa 
eg. Ergonomic factors and student learning (21(2) May 1990: 132-138) 

COMPUTERS AND EDUCATION: AN INTERNATIONAL JOURNAL 
[UK] 4pa 

eg. A package of computer programmes for teaching experimental physiology and 
medical statistics (13(3) 1989: 219-222) 

EDUCATION RESEARCH AND PERSPECTIVES 

[University of Western Australia] 2pa 
eg. A procedure for screening applicants to a tertiary institution (14(2) Dec 1987: 40- 

48) 

EDUCATIONAL & TRAINING TECHNOLOGY INTERNATIONAL 
[Association for Educational and Training Technology (UK)] 4pa 

eg. Educational technology applied to medical training: success and failure in a 
cooperative venture in the Federal Republic of Germany (26(2) 1989: 1 13-121) 

EDUCATIONAL MEDIA INTERNATIONAL 

[International Council for Educational Media (France)] 4pa 
eg. Uses of video in teacher training (23(4) Dec 1986: 133-177) 

EDUCATIONAL PHILOSOPHY AND THEORY 

[Philcsophy of Education Society of Australasia] 2pa 
eg. Science education (20(2) 1988: 1-81) 

EDUCATIONAL RESEARCH 

[National Foundation for Educational Research (UK)] 3pa 
eg. Managing school departments and faculties; towards better practice (31(2) June 

1989: 98-112) 

EDUCATIONAL RESEARCHER 

[American Educational Research Association] 9pa 
eg. Educational assessment (18(9) Dec 1990: 1-32) 
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EDUCATIONAL REVIEW 
[UK]3pa 

eg. The management of curriculum time as it relates to student engaged time (42(1) 
1990: 13-23) 

EUROPEAN JOURNAL OF EDUCATION: RESEARCH, DEVELOPMENT AND 
POLICIES 
[UK]4pa 

eg. Medical education in the European Communities: moving towards 1993 and 
beyond (24(4) 1989: 399-410) 

HERDSA NEWS 

[Higher Education Research and Development Society of Australasia] 3pa 
eg. rroblem-based learning at Lincoln Institute of Health Sciences (9(2) July 1 987: 3- 
6) 

HIGHER EDUCATION: THE INTERNATIONAL JOURNAL OF HIGHER 
EDUCATION AND EDUCATIONAL PLANNING 
[Holland] 8pa 

eg. Assessment in a problem-based medical curriculum (19(2) 1990: 123-146) 

HIGHER EDUCATION QUARTERLY 

[Society for Research into Higher Education (UK)] 4pa 
eg. Loans for learning: the loans versus grants debate in international perspective (43( 1 ) 

Winter 1989:76-87) 

HIGHER EDUCATION RESEARCH & DEVELOPMENT 

[Higher Education Research and Development Society of Australia] 2pa 
eg. Students' perceptions of priorities in nurse education (8(2) 1989: 1 1 7-1 27) 

HIGHER EDUCATION REVIEW 
[UK]3pa 

eg. Training for teamwork in health care (18(2) Spring 1986: 5-20) 

INTERNATIONAL JOURNAL OF SCIENCE EDUCATION 

[UK (Formerly European Journal of Science Education)] 4pa 
eg. Students' perceptions of the concept of health (7(3) July-Sept 1985: 307-323) 

INTERNATIONAL REVIEW OF EDUCATION 

[UNESCO Institute for Education (Germany)] 4pa 
eg. Sports and physical education (35(1) 1989: 3-124) 
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JOURNAL OF BIOLOGICAL EDUCATION 

[Institute of Biology (UK)] 4pa 
eg. Biology and ethics (24(1) Spring 1990: 18-22) 

JOURNAL OF COMPUTER-BASED INSTRUCTION 

[Association fortheDevelopmentof Computer-Based Instructional Systems (USA)] 
4pa 

eg. Medical students' attitudes toward the use of microcomputers as instructional tools 
(16(3) Summer 1989: 90-94) 

JOURNAL OF EDUCATION POLICY 
[UK]4pa 

eg. Institutional amalgamation in higher education (in Australia and UK) (4(1) 1989: 
23-37) 

JOURNAL OF EDUCATIONAL PSYCHOLOGY 

[American Psychological Association] 4pa 
eg. A social cognitive view of self-regulated academic learning (8 1(3) Sept 1989: 329- 

339) 

JOURNAL OF EDUCATIONAL TECHNOLOGY SYSTEMS 

[Society for Applied Learning Technology (USA)] 4pa 
eg. Computer-assisted instruction in health professions education: a review of the 

published literature (18(2) 1989: 83-101) 

JOURNAL OF HIGHER EDUCATION 

[American Association for Higher Education] 6pa 
eg. Comparing educational accreditation standards in selected professional fields (inc. 

nursing and pharmacy) (60(1) Jan-Feb 1989: 1-20) 

JOURNAL OF PHILOSOPHY OF EDUCATION 

[Philosophy of Education Society of Great Britain] 2pa 
eg. Health promotion and health education (22(1) 1988: 3-1 1) 

JOURNAL OF RESEARCH IN SCIENCE TEACHING 

[National Association for Research in Science Teaching (USA)] 9pa 

eg. Methodological limitations for the use of expert systems techniques in science 
education research (27(1) Jan 1990: 69-77) 

JOURNAL OF TERTIARY EDUCATIONAL ADMINISTRATORS 
[Australian Institute of Tertiary Educational Administrators] 2pa 
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eg. Is ignorance bliss? The findings of the Women at Work Study Group at Lincoln 
Institute of Health Sciences (9(1) May 1987: 33-42) 

MUSLIM EDUCATION QUARTERLY 

[Islamic Academy (Cambridge, UK)] 4pa 
eg. Islamic medical education: a way of fostering the Islamic Code of Medical Ethics 

(3(2) 1986: 57-75) 

OPEN LEARNING: THE JOURNAL OF OPEN AND DISTANCE LEARNING 
[UK]3pa 

eg. A comparative study of the effect of tutorial audio-tapes in teaching elementary 
statistics (5(1) Feb 1990: 53-56) 

PHYSICS EDUCATION 

[Institute of Physics (UK)] 6pa 
eg. Medical physics undergraduate degree courses at university (24(4) July 1 989: 227- 

231) 

RESEARCH IN SCIENCE & TECHNOLOGICAL EDUCATION 
[UK]2pa 

eg. Physics in medical diagnosis: implementation and evaluation of an applied science 
unit for high school (7(1) 1989: 93-105) 

REVIEW OF EDUCATIONAL RESEARCH 

[American Educational Research Association] 4pa 
eg. Teaching as clinical problem solving: a critical examination of the analogy and its 

implications (58(4) Winter 1988: 482-505) 

SIGCUE OUTLOOK 

[Association for Computing Machinery Special Interest Group on Computer Uses 

in Education (USA)] 4pa 
eg. Social implications of computers: ethical and equity issues (20(1) Fall 1988: 100- 

114) 

SOCIOLOGY OF EDUCATION 

[American Sociological Association] 4pa 
eg. Standardised testing as an interactional phenomenon (63(2) April 1 990: 83- 1 01 ) 

STUDIES IN CONTINUING EDUCATION 
[Australia] 2pa 

eg. Australian higher education institutions and the provision of continuing education 
(11(2) 1989: 142-160) 
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STUDIES IN EDUCATIONAL EVALUATION 
[UK]3pa 

eg. Educational evaluation in Australia (15(1) 1989:1-159) 

STUDIES IN HIGHER EDUCATION 

[Society for Research into Higher Education (UK)] 3pa 
eg. Identifying medical school applicants from ethnic minorities (15(1) 1990: 57-73) 

STUDIES IN SCIENCE EDUCATION 

[Centre for Studies in Science and Mathematics Education, University of Leeds 
(UK)] 2pa 

eg. Health education and science education: changing roles, common goals? (13, 1986: 
1-26) 

TEACHING OF PSYCHOLOGY 

[American Psychological Association] 4pa 
eg. An cxperientially- and community-oriented graduate course in mental health 

administration (14(1) Feb 1987: 51-53) 

TEACHING SOCIOLOGY 

[American Sociological Association] 4pa 
eg. Teaching medical sociology in medical schools (18(3) July 1990: 319-328) 

UNICORN 

[Australian College of Education] 4pa 
eg. Holistic nursing curricula: towards a reconstruction of health and nursing (16(2). 
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Annals of Community-Oriented Education 
1992, Vol. 5, 331-390 
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FROM THE LITERATURE 



Some Recent Books 



District Management 

Cost analysis inprimary healthcare: A training manual for programme managers. (1990). 

Geneva, Switzerland: World Health Organization. 
Manual of epidemiology for district health management Vaughan, J.P., & Morrow, R.H. 

(Eds.). (1990). Geneva, Switzerland: World Health Organization. 
Medical administration for frontline doctors. Pearson, C.A. (1990). London, United 

Kingdom: FSG Communications Ltd. 
Towards a healthy district: Organizing and managing district health systems based on 

primary health care. Tarimo, E. (1991). Geneva, Switzerland: World Health 

Organization. 

On being in charge: A guide to management in primary health care. Second edition. 
McMahon, R., Barton, E., & Piot, M. (1992). Geneva, Switzerland: World Health 
Organization. 

Education 

Clinical education and the doctor of tomorrow. Gastel, B., & Rogers, D. (Eds.) (1989). 

New York, NY: New York Academy of Medicine. 
Critical thinking: The future of undergraduate medical education. A study by the King's 

Fund Centre in collaboration with St. Bartholomew's Hospital Medical School. Towle, 

A. (1991). London, United Kingdom: King's Fund Centre. ISBN 1 85717 0121. 
Executive skills for medical faculty. Whitman, N., Weiss, E., & Bishop, F.M. (1989). Salt 

Lake City, UT: University of Utah, School of Medicine. 
Innovation in medical education: An evaluation of itspresent status. Nooman, Z.M., Schmidt, 

H.G., & Ezzat, E.S. (Eds.) (1990). New York, NY: Springer Publishing Co. 
Instructional design: Principles and applications. Second edition. Briggs, L. J., Gustaf son, 

K.L., Tillman, M.H. (1990). Englewood Cliffs, NJ: Educational Technology 

Publications. 

International consultation on health manp wer education for health for all. Summary report 

of consultation, Venice, December 1989. (1990). Copenhagen, Denmark: European 

Regional Office, World Health Organization. 
International medical communication in English. Maher, J.C. (1990). Edinburgh, United 

Kingdom: Edinburgh University Press. 
Inter-professional collaboration and education: An annotated bibliography. Toase, M. 

(1991). London, United Kingdom: CAIPE do London School of Economics. 
Learning to teach in higher education. Ramsden, P. (1992). London, United Kingdom: 

Routledge. 
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Revitalising medical education: McMaster Medical School, the early years 1965-1974. 

Spaulding, W.B. (1991). Philadelphia, PA: B.C. Decker. 
South Australian health sciences education review. (1991). Adelaide, Australia: Office of 

Tertiary Education. 

Teaching skills: Aguideforteachersof health workers. Ewin, C.E. (1988). Sydney, Australia: 
University of New South Wales. 

Environment 

Environmental health in urban development. (1991). Technical Report Series No. 807. 

Geneva, Switzerland: World Health Organization. 
Sustainable development: An imperative for environmental protection. Report by a 

Commonwealth Group of Experts. (1991). London, United Kingdom: Commonwealth 

Secretariat. 

Health Promotion 

Ethics in health education. Doxiadis, S. (1990). Chichester, United Kingdom: John Wiley 
& Sons. 

Evaluating health promotion: A health worker's guide. Hawe, P., Degeling, D., & Hall, 

J. (1990). Sydney, Australia: MacLennan & Petty. 
Health behavior and health education: Theory, research and practice. GJanz, K., Lewis, 

R, & Rimers, B. (Eds.) (1990). San Francisco, CA: Jossey Bass Publishers. 
Health measurement scales: Apraeticalguidetotheirdevelopmentanduse. Strainer, D.L., 

& Norman, G.R. (1991). Oxford, United Kingdom: Oxford University Press. 
Health promotion: A research book. Anderson, R., & Kickbus, I. (1990). Copenhagen, 

Denmark: European Regional Office, World Health Organization. 
Learning nutrition in medical schools. Report of WHO workshop, Warsaw, April 1989. 

(1990) . Copenhagen, Denmark: European Regional Office, World Health Organization. 

Postgraduate and Continuing Education 

Learning general practice. Third edition. (1991). Edinburgh, United Kingdom: Scottish 

Council for Postgraduate Medical Education. (Slim guide for trainees on how to learn 

and plan their learning) 
Maintaining professional competence: A survey of the role of professional bodies in the 

development of credit-based continuing professional development courses. Vaughan, 

P. (1991). Hull, United Kingdom: University of Hull. 
Manual of use and interpretation of pathology tests. McPherson, J., & Thomas, D.W. 

(Eds.) ( 1990). Surrey Hills, Australia: Royal College of Pathologists of Australia. ISBN 

1 86252 9558. (Pocket book guide for selection of appropriate tests and their 

interpretation) 

Report of seminar on community-based postgraduate programme in family medicine in 
Malaysia and recommendations for its implementation. Sharifah H. Shahabudin. (Ed.) 

( 1 99 1 ) . Kuala Lumpur, Malaysia: Faculty of Medicine, University Kebangsaan Malaysia. 
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Report of seminar Towards quality health care through continuing medical education. 

Edariah Abu Baker. (Ed.) (1990). Kuala Lumpur, Malaysia: Faculty of Medicine, 

University Kebangsaan Malaysia. 
Second consultation with leading medical practitioners: Continuing medical education. 

(1991). Geneva, Switzerland: World Health Organization. 
System of continuing education: Priorities to district health personnel. (1990). Technical 

Report Series No. 803. Geneva, Switzerland: World Health Organization. 

Primary Care 

A textbook of family medicine. McWhinney, I.R. (Ed.) (1989). Oxford, United Kingdom: 

Oxford University Press. 
Community-oriented primary care. From principle to practice. Nutting, P. A. (Ed.) ( 1 990). 

Albuquerque, NM: University of New Mexico Press. 
The art ofgeneralpractice. Morrell, D. ( 1 99 1 ) . Oxford, United Kingdom: Oxford University 

Press. 

Problem-Based Learning 

Glossary: A list of frequently used concepts in the field of problem-based learning. Frijns, 

P., de Grave, W., & Wolfhagen, I. (1991). Maastricht, The Netherlands: Faculty of 

Medicine, University of Limburg. 
New directions for medical education: Problem-based learning and community-oriented 

medical education. Schmidt, H.G., Lipkin, M., de Vries, M.W., & Greep, J.M. (Eds.) 

(1989). Berlin/Heidelberg, Germany: Springer Verlag. 
Problem-based learning: Perspectives from the Maastricht experience. Van der Vleuten, 

C, & Wijnen, W. (Eds.) (1990). Amsterdam, The Netherlands: Thesis. 
Small group teaching: A trouble-shooting guide. Tiberius, R.G. (Ed.) (J 989). Toronto, 

Canada: OISE Press. 

The challenge of problem-based learning. Boud, D., & Feletti, G. (Eds.) (1991). London, 
United Kingdom: Kogan Page. 

Professional Skills 

Counselling and communication in health care. Davis, H., & Fallowfield, L. (eds.) (1991 ). 

Chichester, United Kingdom: John Wiley & Sons. 
Developing clinical problem solving skills: A guide to more effective diagnosis and 

treatment. Barrows, H.S. & Pickell, G.C. (1991). New York, NY: W.W. Norton & Co. 
Developing communication and counselling skills in medicine. Corney, R. (Ed.) (1991). 

London, United Kingdom: Routledge. 
Handbook of clinical teaching: Exercises and guidelines for health professionals who 

teach patients, train staff or supervise students. Watts, N.T. (1990). Edinburgh, United 

Kingdom: Churchill Livingston. 
Talking with patients: A basic clinical skill. Myerscough, P.R. (1989). Oxford, United 

Kingdom: Oxford University Press. 
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The clinical encounter: A guide to the medical interview and case presentation. Billings, 
J.A., & Stoeckle, J.D. (1989). Chicago, IL: Mosby Year Book Medical Publishers. 

Research 

The role of research and information systems in decision making for the development of 
human resources for health. (1990). Technical Report Series No. 802. Geneva, 
Switzerland: World Health Organization. 

Writing up qualitative research. Wolcott, H.F. (1990). Newbury Park, CA: Sage. 
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Some Journals and Newsletters 

CAIPE Newsletter. Centre for the Advancement of Interprofessional Education in Primary 
Health and Community Care, c/o London School of Economics, London WC2A 2AE, 
United Kingdom. 

Change in Medical Education. A new network for medical education. King* s Fund Centre, 

126 Albert Street, London NW1 7NF, United Kingdom, 
Health Action. Appropriate Resources & Technologies Action Group, 1 London Bridge 

Street, London SE1 9SG, United Kingdom. 
Joint Medical Newsletter. Faculty of Medicine, University of Newcastle, N.S.W., 2308, 

Australia. 

Journal of the Bahrain Medical Society. Editor, P.O. Box 26136, Manama, State of 
Bahrain. 

KIT Newsletter (The International Newsletter of the Royal Tropical Institute in Amster- 
dam). Secretariat, KIT, Mauritskade 63, 1092 AD Amsterdam, The Netherlands. 

Medizinische Ausbildung. Journal of the Gesellschaft fiir medizinische Ausbildung, c/o 
medizinische Fakultat, Universitat MUnster, Domagkstrasse 3, D-4400 MUnster, 
Germany. 

Network News: Health Learning. HLM Clearing House, Division of Development of 

Human Resources for Health, WorldHealth Organization, 121 1 Geneva27, Switzerland. 
Newsletter. Action in International Medicine. Secretariat, Windeyer Building, 46 Cleveland 

Street, London W1P 6DB, United Kingdom. 
Newsletter. European Network for Development of Multiprofessional Education in Health 

Sciences. Secretariat, Department of General Practice, Postgraduate Medical School, 

Barrack Road, Exeter, Devon DC2 SDW, United Kingdom. 
Newsletter. Network of Community-Oriented Educational Institutions for Health Sciences. 

Network Secretariat, Faculty of Medicine, University of Limburg, P.O. Box 616, 6200 

MD Maastricht, The Netherlands. 
Newsletter. The Aga Khan Health Services. Secretariat, Public Affairs Department, The 

Aga Khan University Medical Centre, Stadium Road, P.O. Box 3500, Karachi-74800, 

PaHstan. 

NGO-WHO Newsletter. World Health Organization, 121 1 Geneva 27, Switzerland. 
Pedagogue. Perspectives on Health Sciences Education. The Editor, Program for Educational 

Development, Room 3NS 1 Health Sciences Centre, McMaster University, 1200 Main 

Street West, Hamilton, Ontario, Canada L8N 3Z5. 
Prioritas Kesehetan/Health Priorities. Secretariat, Faculty of Medicine, Gadjah Mada 

University, Sekip 55281 Yogyakarta, Indonesia. 
Research in Distance Education. The Centre for Distance Education, Athabasca University, 

P.O. Box 10.000, Athabasca, Alberta, Canada TOG 2R0. 
Santi. Cahiers d'ltudes et de recherches francophones. John Libbey Eurotext. 6, rue 

Blanche, 92120 Montrouge, France. 
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Teaching and Learning News. Eastern Mediterranean Regional Office, World Health 

Organization, P.O. Box 1517, Alexandria, Egypt 
WONCA News. World Organization of Family Doctors. Secretariat, 4th Floor, 70 Jolimont 

Street, Jolimont, Victoria 3002, Australia. 
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Some Recent Papers 

With the generous consent of the Editors of Academic Medicine (formerly the Journal of 
Medical Education), Medical Education, Medical Teacher, and Teaching and Learning in 
Medicine we have reproduced the abstracts of a number of papers that may be of particular 
interest to readers of the Annals. Where it seemed of especial help to readers, who may not 
have easy access to comprehensive library facilities, we have added the references which 
were given at the end of a paper. 

We have arranged the S3 abstracts under the following headings: 
Administration, Assessment, Community-Oriented and General Practice, Educational 
Methods, Evaluation, Problem-Based Learning, Problem-Solving, Skills Training, Students 9 
Learning, Research, and Teaching. 

The addresses for ordering these journals are: 

Academic Medicine: (formerly Journal of Medical Education): Association of American 
Medical Colleges, 1 Dupont Circle, N.W., Washington, DC 20036, USA 

Medical Education: Blackwell Publishing Co., P.O. Box 87, Osney Mead, Oxford, 0X2 
ODT, United Kingdom 

Medical Teacher Carfax Publishing Co., P.O. Box 24, Abington, 0X14 2UE, United 
Kingdom 

Teaching and Learning in Medicine: Lawrence Erlbaum Associates, 365 Broadway, 
Hillsdale, NJ 07642, USA. 

Administration 

Managing the Initial Period of Implementation of Educational Change. Shahabudin, 
H., & Safiah, N. (Malaysia). (1991). Medical Teacher, 73, 3. 
Three years ago the University Kebangsaan Malaysia medical school changed its 
curriculum from the traditional discipline based curriculum to the integrated organ- 
system approach. Once change was effected a process of 'rcfreezing' had to be initiated 
whereby new reponses had to be reintegrated into the ongoing personality or emotional 
relationships of important people so that the change process will endure and become 
stable. During this refreezing process the faculty encountered several problems which 
could thwart further development of the new curricula if left unresolved. The nature of 
the problems seemed to indict that curricular change involves more than just efforts at 
bettering the what and ways of student learning and assessment. A lot of energy was also 
spent on keeping things going, keeping people motivated, making sure the work was 
done (at least as well as it has in the past), looking for better ways to do things, weighing 
new solutions and to be alert to new problems. In ensuring the continuance of change 
it was important to ensure, from the outset the institutional! sation of policies, programmes, 
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procedures and practices for continuing reward, routinization, structural integration 
into the system, continuing evaluation and providing for continuing maintenance. 

A Model of a Microcomputer Database System to Assist Medical Schools in Recruiting 
and Retaining Students. Furutani, J.C. (U.S.A.). (1991). Academic Medicine, 66, 
114-118. 

A model for a microcomputer database to ad a school's efforts to recruit and retain 
students is presented. The model is base upon a working system developed by the author 
at the University of Colorado Health Sciences Center. The article discusses data needs 
for adequate records of the recruitment and retention efforts, especially those concerning 
minority students, and how these records can be used to administer and evaluate 
programmes. The model provides a framework for microcomputer data files, record 
fields, relationships, and system development and maintenance considerations. 

Estimating Impacts on Developing Countries of the Decrease in U.S. Training 
Opportunities for Foreign Medical Graduates. Harrington, W.J. Sr., Gotuzzo, E., 
Vial, S.U., Restrepo, J., Baldi, J., Young, P.M., Defillo-R, M., Guderian, R, & 
Harrington, W.J. Jr. (U.S.A). (1991). Academic Medicine, 66, 707-709. 
Between 1973 and 1983, the number of foreign nationals from developing nations who 
entered the United States for graduate medical education decreased by approximately 
90%. Many of those who would have studied in the United States if this decrease had 
not occurred would have returned home to serve their countries. To estimate the impact 
of this loss, a survey was conducted in six major cities in Latin America between 1983 
and 1989. Selected local medical students interviewed 554 physicians who had 
returned home after U.S. training and 60 of their classmates who had not trained there. 
The findings indicate that the returned physicians had given approximately twice as 
much time to teaching, research, and medical administration as did those who had not 
left home. The authors maintain that this and related findings show how the curtailment 
of opportunities for training foreign nationals in the United States is detrimental to both 
the aspirations of developing nations and the influence of the United States in world 
affairs. 



Assessment 

Rethinking Critical Issues in Performance Assessment Friedman, M., & Mennin, A.P. 
(U.S.A.). (1991) Academic Medicine, 66, 390-395. 

The recent interest of medical schools and licensure organisations in establishing 
performance assessment methods in medical education presents new challenges to 
medical educators. The problems encountered in establishing the reliability and 
validity. The authors examine the relationship between Classical Test Theory and job 
performance. They discuss several critical issues: the meanings of reliability and validity 
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in performance assessment and the balance between them (including simple versus 
complex behaviours), stability of performance, specific versus generic abilities, and the 
role of experts in clinical performance assessment The authors call for a critical 
appraisal of apply ing Classical Test Theory to the assessment of job-related behaviours. 

Pitfalls in the Pursuit of Objectivity: Issues of Reliability. Van der Vleuten, CRM., 
Norman, G.R., &DeGraaf, E. (Canada, The Netherlands). (1991). Medical Education, 
25, 110-118. 

Objectivity has been one of the hallmarks in the assessment of clinical competence in 
recent decades. A consistent shift can be noticed in which subjective measures are being 
replaced by objective measurement methods. In the transition from subjective to 
objective methods trade-offs are involved, both in the effort expended and in the range 
of behaviours assessed. The issue of the presumed superiority of objective measures is 
addressed in two successive papers. 

In this paper a distinction is made between objectivity as a goal of measurement, 
marked by freedom of subjective influences in general, and objectivity as a set of 
strategies designed to reduce measurement error. The central claim of this paper is that 
these two approaches to assessment do not necessarily coincide. By reviewing a 
number of studies comparing subjective and objectified measurement methods, the 
claim of the supremacy of the latter with respect to reliability is discussed. 
The results of these studies indicate that objectified methods do not inherently provide 
more reliable scores. Objectified methods may even provide unwanted outcomes, such 
as negative effects on study behaviour and triviality of the content being measured. The 
latter issues, related to validity, efficiency and acceptability, are discussed in a second 
paper. 
References 

Bender, W., Hiemstra, Scherpbier, A.J.J.A., & Zwierstra R.P. (1990). Teaching and 
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Pitfalls in the Pursuit of Objectivity: Issues of Validity, Efficiency and Acceptability. 

Norman, G.R., Van der Vleuten, C.P.M., & De Graaff, E. (Canada, The Netherlands). 
(1991). Medical Education, 25, 119-126. 

In a previous article the distinction is made between objectivity and objectification . 
Objectivity is considered a generic goal of measurement, marked by freedom of 
subjective influences in general, whereas the latter term is used to describe strategies 
to reduce measurement error. A survey of several studies indicated that objectified 
methods are not intrinsically more reliable than subjective measures. 
In this paper the consequences of objectification are analysed for issues related to 
validity, efficiency, transparency, and effect of these methods on students and teacher. 
Several studies comparing objectified and subjective methods are surveyed for this 
propose. 

The studies indicate that - as in the previous article on reliability * objectification and 
objectivity are not identical, and that there are many pitfalls in the objectification of 
measurement procedures. As a consequence, it is argued that objectified methods 
should not exclusively be chosen on the basis of their unconditional appeal to 
objectivity, but that the application of measurement methods should follow the specific 
purpose of the testing situation. In the context of the testing situation, arguments against 
and in favour of objectification should be weighted, and trade-offs are to be evaluated. 
The outcome of this evaluation may vary from situation to situation, and from 
institution to institution. 
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The performance of foreign medical graduates on multistation standardised patient- 
based tests was used to determine the validity and generalisability of global ratings of 
their clinical competence made by expert examiners. Data were derived from the 
entrance examinations of the 1989 and 1990 applicants to the Ontario Pre-Internship 
Program and the exit examination of 24 participants from the 1989 cohort For each 
candidate, the examiners completed a detailed checklist and two five-point global 
ratings dealing with the candidate's approach to the patient's problem and attitude 
toward the patient. Generalisability coefficients for both ratings were satisfactory and 
stable across cohorts. Construct validity of the global ratings was demonstrated by 
comparing entry and exit ratings and by evidence of significant and positive correlations 
between the global ratings and total test scores. Tentative evidence of criterion validity 
of the global ratings was demonstrated. These findings suggest that global ratings by 
expert examiners can be used as an effective form of assessment in multistation 
standardised patient examinations. 

A Multi-Institutional Trial of an Objective Structured Clinical Examination. Fetrusa, 
E.R., Blackwell, T.A., Carline, J., Ramsey, P.G., McGaghie, W., Colindres, R., 
Kowlowitz, V., Mast, T.A, & Soler, N. (U.S. A.). (1991). Teaching and Learning in 
Medicine, 2, 86-94. 

This study investigated the feasibility of implementing the same 10 cases in an 
objective structural clinical examination (OSCE) to evaluate medical students' clinical 
skills following the medicine clerkship at four geographically dispersed medical 
schools. Results indicated that a clinically equivalent, standardised test of clinical 
performance could be developed at multiple schools. Generalisability of student scores 
ranged from .26 to .50. Overall performance averaged 63%, with case scores ranging 
from 74% for a history of recurrent urinary tract infections to 52% for fever and cough. 
Significant differences were found among schools for individual cases, but performance 
from any one school was not consistently highest. Clinical skills scores ranged from 
72% for physical examination technique to 58% for initial management, with varying 
significant differences among schools, suggesting inconsistent clinical training within 
and among schools. Performance was not higher for students examined later in the third 
year. Participant reactions were generally favourable. Results support the feasibility of 
implementing standardised OSCEs that would provide importantdataon the performance 
of graduates and on the adequacy of clinical education from different medical schools. 

The Influence of the Introduction of Objective Structured Practical Examinations 
in Physiology on Student Performance at King Faisal University Medical School. 

Dissanayake, A.S., Ali, B.A., & Nayar, U. (Saudi Arabia). (1990). Medical Teacher, 
12, 3-4. 

The Department of Physiology at the College of Medicine and Medical Sciences, King 
Faisal University ( KFU) had serious concerns about the effectiveness of these practicals. 
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A new form of evaluation, the objective stnictured practical examination (OSPE), was 
introduced in June 1987. At KFU, the introduction of OSPE has led to a marked 
improvement in the mean scores for the laboratory component of the final examinations 
in the physiology courses. Such an improvement was not consistently observed in the 
other components of the examinations. Performance in stations evaluating skills was 
in general better than in those testing analytical and interpretive abilities. 
Female students performed significantly better in OSPE than male students in 4/5 and 
3/5 physiology I and II examinations. Such gender differences were not observed in the 
written examinations. 

Reliability and Feasibility of Measuring Medical Interviewing Skills: The Revised 
Maastricht History-Taking and Advice Checklist Van Thiel, J., Kraan, H.F., & Van 
der Vleuten, CP.M. (The Netherlands). (1991). Medical Education, 25, 224-229. 
Medical interviewing skills are integral to good medical care. In cider to measure these 
skills an instrument has been developed, called the Maastricht Histo;y-Taking and 
Advice Checklist (MAAS). It has been studied with regard to interrater reliability and 
validity. In this study a revised version of the MAAS (MAAS-R), a check-list of 
concrete interview behaviour, has been investigated concerning feasibility and reliability 
for examination purposes. Audio-recordings were obtained of 24 doctors, each 
interviewing eight different standardised patients. The recordings were independently 
scored by three general practitioners trained in using the MAAS-R. The results of 
generalisability analysis, considering the influences of doctors, cases and raters, are 
encouraging. In order to overcome case-specificity, feasible and reliable measurement 
can be accomplished with 8-10 cases in 2-2 1/2 hours of testing time, each case being 
scored by a different rater. Reliability improves considerably if assessment is restricted 
to basic interviewing skills. 

A Review of the Validity and Accuracy of Self-Assessments in Health Professions 
Training. Gordon, MJ. (U.S.A.). (1991). Academic Medicine, 66, 762-769. 
Valid self-assessment is fundamental to continuing professional competence but is 
seldom explicitly taught in health professions training. This review analysed 16 
scholarly articles published between January 1970 and February 1990 (14 articles 
regarding health professions trainees, and four concerning college students or graduate 
trainees) in which it was possible to compare performance as self-assessed by trainees 
with performance as assessed by experts or objective tests. The validity of self-assessed 
performance was found to be low to moderate and did not improve with time in 
conventional health professions training programs. Self-assessed performance seemed 
closely related to generalised self-attributions and was minimally influenced by 
external feedback in the form of test scores, grades, or faculty assessments. In five 
programs, emphasizing explicit self-assessment goals and training strategies, moderate- 
to-high validity outcomes or improvements over time were demonstrated. Much of 
what passes for self-assessment in training seems the exercise of an underdeveloped 
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skill, but effective training to improve validity and accuracy is available and feasible. 
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Test Security in Examinations That Use Standardised-Patient Oases at One Medical 
School. Colliver, J. A., Barrows, H.S., Viet Vu, N., Verhulst, S J. f Mast, T. A., & Travis, 
T.A. (U.S.A.). (1991). Academic Medicine, 66, 279-282. 

The use of performance-based examinations consisting of standardised-patient (SP) 
cases has increased greatly in recent years. These examinations are typically long and 
thus require the presentation of the same SP cases to several consecutive examinee 
groups. Consequently, concerns have arisen about the potential for violations of test 
security whereby students who were tested early in the examination period pass on 
information to students tested later. These concerns are addressed using data from the 
SP-based examinations administered to five classes (1986-1990) of senior medical 
students at Southern Illinois University School of Medicine. Because of the length of 
the examinations, each class was randomly divided into five groups and the examination 
was administered to one group at a time, requiring three days of testing time per group 
anil three weeks of testing time per class. The results showed no consistent, systematic 
changes ?n case means across the five groups tested at different times throughout the 
examination periods, and thus provide no evidence of serious, widespread violations 
of test security. 

TheAccuracy of Standardised Patient Presentation- Tamblyn,R.M.,Klass,D J., Schnabl, 
G.K., & Kopelow, M.L. (Canada). (1991). Medical Education, 25, 100-109. 
The accuracy of standardised patient clinical problem presentation was evaluated by 
videotape rating of a random sample of 839 student-patient encounters, representing 88 
patients, 27 cases and two university test sites. Patient-Student encounters were 
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sampled from a collaborative inter-university final-year clinical examination of fourth- 
year medical students which was conducted at the University of Manitoba and Southern 
Dlinois University in 1987 and 1988. The accuracy, replicability and portability of 
standardised patient cases were evaluated. The average accuracy of patient presentation 
was 90.2% in 1987 and 93.4% in 1988. Perfect accuracy scores were obtained by 15 
patients; however, 1 1 patients had average scores below 80% with the accuracy of 
presentation in some encounters being as low as 30% . There were significant differences 
in the accuracy score achieved by patients trained together for the same case in 6 of 35 
possible comparisons. There was also a systematic trend for patients trained at Southern 
Illinois to be more accurate in their presentation than patients trained at the university 
of Manitoba. These differences were significant in 5 of the 15 cases used in the 
examination. 
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Sources of Unreliability and Bias InStondanIised-PatientIUting.Tamblyn,M.,Klass, 

J., Schnabl, K., & Kopelow, L. (Canada, U.S.A.). (1991). Teaching and Learning in 
Medicine, 3, 74-85. 

In tests of clinical competence, standardised patients (SPs) can be used to present the 
clinical problem and rate actions taken by the examinee in the patient encounter. Both 
these aspects of the 'test" have the potential to contribute to unreliability and bias in 
measurement. In 1987, two universities collaborated to develop and execute the same 
SP test to clinical clerks in their respective institutions. This provided us with the 
opportunity to evaluate rating bias attributable to test site and three sources of rating 
unreliability within the same population of raters: those attributable to inconsistence is 
within the same rater (within-rater reliability), those attributable to inconsistencies 
between two raters trained in the same test site (between-raters reliability-same site), 
and those attributable to inconsistencies between two raters trained in different test sites 
(between-raters reliability-different sites). A stratified random sample of 537 of the 
2,560 examinee-patient encounters that occurred in the inter-university examination 
was videotaped, providing equivalent representation of the 1 6 cases used in the test and 
the two universities. Videotaped encounters from both universities were rated by 44 
SPs who presented and rated the case during the examination. Videotaped and 
examination ratings were used to estimate systematic rating bias and the three types of 
rater reliability. Overall, rater reliability of individual items and overall encounter score 
were fair to good (.37 to .52). Consistent with these results, raters within cases 
accounted for 20% of the observed variance in student scores. Within-rater reliability 
was better than both types of between-raters reliability. Rater agreement was not 
influenced by test site, but systematic differences in score were present between test 
sites. Site 1 raters scored the same students, on average, 6.7% lower than Site 2 raters. 
These differences had an impact on the proportion of students who would have failed 
the check list portion of the test. In Site 1 , 50% of the students rated had data-collection 
scores below 60%, whereas, in Site 2, only 33% had scores below the 60% cutoff. The 
implications of these findings for single- and multi-site SP-based tests of competence 
are explored, and additional areas for research are identified. 
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Effects of Using Two or More Standardised Patients to Simulate the Same Case on 
Case Means and Case Failure Rates. Colli ver, J.A., Robbs, R.S., & Vu, N.V. (U.S.A.). 
(1991). Academic Medicine, 66, 16-618. 

The effects of using two or more standardised patient (multiple SPs) to simulate the 
same case in a performance-based examination were studied at the case level by 
comparing case means and case failure rates for multiple SPs simulating the same case, 
using data from the classes of 1988, 1989, and 1990 at the Southern Illinois University 
School of Medicine. For total scores and scores on the students* written answers, the 
effects on means and failure rates were negligible and could be explained as due to 
sampling error. For scores on the checklists completed by the SPs, there were more 
significant differences than would be expected by chance alone, even though the 
number of significant differences was relatively small. The results demonstrate a need 
for caution in the interpretation of scores obtained from a case checklist completed by 
multiple SPs, particularly in regard to making pass-fail decisions. 
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Williams, R.G., et al. (1987). Direct standardised assessment of clinical competence. 
Medical Education, 21, 482-489. 

Reliability and Efficiency of Components of Clinical Competence Assessed with Five 
Performance-based Examinations Using Standardised Patients. Colliver, J. A., Vu, 
N.V., Markwell, S.J., & Verhulst, S.J. (U.S.A.). (1991). Medical Education, 25, 303- 
310. 

The present study was conducted to provide in-depth information on the reliabilities of 
measures of the separate components of clinical competence (e.g. data collection, test 
interpretation, diagnosis, etc.) assessed by a performance-based examination consisting 
of standardised patient cases administered to five classes of senior medical students at 
Southern Illinois University School of Medicine. In general, the reliabilities of the 
competencies as they were actually measured on the examination (using the number of 
cases on which each competency was actually measured) were small, with 54% less 
than 0.30 and 75% less than 0.40. For generalisability coefficients pooled across the 
five classes and projected to a common number of k=10 cases, two of the nine 
competencies had reliabilities in the 40s, a third was close to 0.40, and the remaining 
six were in the low 20s. The number of cases needed for the competencies to achieve 
the recommended reliability of 0.80 ranged from 45 to 170 cases, with six of the nine 
competencies requiring over 100 cases to reach the 0.80 level. The low reliabilities of 
these measures of the components of clinical competence raise serious questions about 
using the scores as indicators of student performance. 

The Use of Patients as Student Evaluators, Henkin, Y., Friedman, M., Bouskila, D., 
Kushnir, D., & Glick, S. (Israel). (1990). Medical Teacher, 12, 279-288. 
Current evaluation of medical students in their clinical clerkships is usually performed 
by their clinical tutors, and emphasizes knowledge, skills and behaviour. However, it 
is not at all certain that the clinical tutors adequately appreciate the behavioural aspects 
of the student-patient interrelationship. Since these interactions are not directly 
observed by the tutors, their evaluation is based largely on second hand information. 
In the present study, patient opinion regarding student behaviour was collected and 
compared to tutors' evaluation of the student. One hundred and eighty one patients and 
31 clinical tutors evaluated 41 medical students during a 6 week internal medicine 
clerkship, using a specially designed patient questionnaire and tutors 5 rating form. An 
analysis of the correlation between tutors' individual rating parameters and the global 
score revealed high correlation for those parameters associated with clinical knowledge 
and skills, but lower correlations for parameters associated with students' relationship 
to patients. Hie study reveals difficulties associated with the evaluation of students by 
patients. The process is time-consuming, and introduces tension between students and 
patients. In addition the patients appeared to be poor discriminators in evaluating 
medical students. Finally, the stucty points out that tutors do not assign sufficient 
importance to the student-patient relationship in the global clerkship score. 
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Observer Evaluation as a Tool for Field Level Evaluation of Performance. Weerakoon, 
P.K., & Fernando, D.N. (Sri Lanka). (1990). Medical Teacher, 12, 31 1-314. 
A field-level assessment of the job performance of a category of health professionals 
was carried out using the technique of non-participant observation of client-health 
worker interactions. Presenting the data so obtained in the form of a 'performance 
profile' enabled identification of areas in which this category of health workers showed 
a lower level of performance, indicating areas where alterations in basic and in-service 
training programmes are required. 

Investigating Whether Teachers Should be Given Assessments of Students Made by 
Previous Teachers. Cohen, G.S ., & Blumberg, P. (Canada) ( 1 99 1 ). Academic Medic * r 
66, 288-289. 

Based upon a consideration of data from the literature, a survey, and a problem-solving 
discussion at a national meeting, the authors suggest the following recommendations 
concerning the provision to faculty of previous assessment information on students. 
Students' problems noted by faculty should be shared with the faculty who subsequently 
teach them. Students must be central to the entire process of disclosing earlier 
assessment and should be encouraged to initiate the process. The data should be 
provided by all appropriate faculty and should be distributed to the student advisor and 
the dean" s office for dissemination to the course director of the next unit. Confidential ity 
must be assured. The rationale for such a policy is discussed. 
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Community-Oriented and General Practice 

Effects of an Expanded Medical Curriculum on the Number of Graduates Practicing 
in a Rural State. Stratton, T.D., Geller, J.M., Ludtke, R.L., & Fickenscher, K.M. 
(U.S.A.). (1991). Academic Medicine, 66, 101-105. 

Li 1973 the University of North Dakota School of Medicine (UNDSM), following the 
national trend toward four-year medical programmes, expanded its previous two-year 
medical school curriculum to include all four years of medical education. It was hoped 
that this change, along with a renewed emphasis on primary care-oriented residency 
training within the state, would encourage medical students to establish practices 
within the state. In 1985 the UNDSM' s Centei for Rural Health mailed questionnaires 
to the 2,230 living graduates of the UNDSM to document a variety of their personal and 
practice characteristics. Based on the responses to the 924 completed questionnaires, 
the authors found that: 1) the students from rural North Dakota were more likely than 
were urban students to practise in rural areas of the state, as were the students with 
primary care specialty training; 2) the alumni completing residencies in North Dakota 
following the curriculum expansion (1976-1985) were more than twice as likely to 
establish practices in North Dakota. It was concluded that recruiting medical students 
(preferably in-state "natives") from rural areas, training them in primary care specialty 
areas, and enabling them to remain in North Dakota for the duration of their medical 
training (including residency training) combined to exert a con- ierable "retaining" 
effect on the UNDSM alumni. 

The Association of Teachers of Preventive Medicine's Recommendations for 
Posfcr&duate Education in Prevention. Collins, T.R., Goldenberg, K., Ring, A., 
Nelson X, & Konen, J. (U.S.A.). (1991). Academic Medicine, 66, 317-320. 
The Cutde to Clinical Preventive Services, prepared in 1989 by the U.S. Preventive 
Services Task Force, assesses the effectiveness of 1 69 Cypes of preventive interventions. 
In 1990, 'die Association of Teachers of Preventive Medicine formed a panel to review 
the guide and recommend ways it could be used to enhance both undergraduate and 
postgraduate medical education. This paper outlines the panel's recommendations of 
the types of knowledge and attitudes on which postgraduate medical education in 
prevention should be built. Detailed recommendations are presented, based on the 
summary findings of the guide, for residency education in prevention. Implementation 
of these recommendations will integrate preventive services into the continuum of 
medical care. These recommendations are presented to achieve the goal of educating 
physicians to approach the total patient, putting the patient's health rather than the 
disease process in the forefront of primary medical care. 
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Implementing the Association ofTeachers of Prevent! ve Medicine's Recommendations 
into the Undergraduate Medical School Curriculum. Altekruse, J., Goldenberg, KL, 
Rabin, D.L., Riegelman R.K., & Wiese, W.H. (U.S.A.), (1991). Academic Medicine, 
66, 312-316. 

In 1989, an expert panel appointed by the Association of Teachers of Preventive 
Medicine proposed minimum curricular content requirements for health promotion- 
disease prevention, including recommendations for timing, duration, and course 
sequencing during medical school. Making clinical preventive medicine an integral 
part of a primary care rotation is a central feature of the proposal. The panel presents 
recommendations for using the Guide to Clinical Preventive Services, which assesses 
the effectiveness of 169 types of prevention interventions, in both undergraduate and 
postgraduate medical education. Recommendations for incorporating the guide into the 
undergraduate medical school curriculum are outlined. The recommendations include 
options for using the guide as part of a curriculum in quantitative skills, in clinical 
preventive medicine, in a primary care rotation, as a health services and community 
dimension curriculum, and as part of continuing self-education. Recognising that 
teaching methods and curriculum structures are varied in preventive medicine, the 
panel designed the recommendations to be adaptable to all medical schools 9 programs. 
The recommendations are aimed at achieving the goal of making preventive medicine 
an integral part of the education, training, and practice of physicians. 
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A Community Pre-registration Year: The Continuation of a Community-Oriented 
Medical School Curriculum. ?ortcT,B.(lsnel).Xl99l). Medical Education, 25, 151- 
154. 

A voluntary community-based pre-registration year (internship) was created for 
graduates of a community-oriented medical school in Beersheva, Israel. Graduates 
were placed in community clinics which had been upgraded to meet the needs of the 
programme through provision of on-site laboratory facilities and specialist consultation, 
and placement of social workers and health education personnel in the clinics. The 
graduates participated in a weekly postgraduate education programme directed to their 
specific needs. There was a high degree of satisfaction with the programmes among the 
trainees, the clinic team members and the patients. Relatively few of the trainees 
indicated a desire to pursue careers entirely devoted to primary care, despite their 
positive evaluation of the pre-registration experience. Though this might have been 
explained by problems encountered in the programme, such as the lack of adequate on- 
site supervision and lack of support from the health service providers, external factors 
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such as a general move of graduates towards mote lucrative sub-specialties and the 
relative lack of status of family practice were probably the more significant reasons. 
Increasing focus on the internship year, in addition to the undergraduate and residency 
periods, is needed to help direct medical education to ambulatory settings. 

Advanced Training in Gerund Practice in the UK for Overseas Postgraduates. Al- 

Bashir, M., Bundrcd, P.E., & Stanley, I.M. (United Kingdom). (1991). Medical 
Education, 25, 60-65. 

The strengths of general practice in the UK as a training environment for overseas 
doctors intending to implement the World Health Organization strategy "Health for All 
by the Year 2000" in primary care are identified. A course of advanced training for 
teachers and administrators of primary care is described and evaluated in terms of 
participants' academic achievements and wider issues, including influence upon the 
development of primary care in their countries of origin. 

Teaching Health Promotion and Diness Prevention to Trainee General Practitioners. 

Hays, R.B. (Australia). (1991). Medical Teacher, 13, 223-226. 
This study demonstrates that the provision of feedback based on the observation of 
videotaped consultations can be used to increase self-awareness of deficiencies in 
health promotion and illness prevention skills. Trainee general practitioners were 
found to have some awareness of deficiencies in this important aspect of general 
practice, and this awareness increased as a result of self-observation and the feedback. 
Performance was self-evaluated to be better at the end of the general practice term than 
at the beginning. 
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Educational Methods 

Teaching Acute Respiratory Infection Using Low-cost Aids* An Experience in 
Pondicherry, South India. Soudarssanane, M.B., Rotti, S.B., & Premarajan, K.C. 
(}n<ii2L)Xl99\). Medical Teacher, 13, 369-370. 

The topic of acute respiratory infections (ARI) was taught to two successive batches of 
medical students in their first clinical year. The low-cost aids in the form of slide set, 
tape recorder and video cassette were used. The knowledge and attitude of students was 
assessed using a p re-test and a post-test, which showed & statistically significant 
improvement. 

The Uses and Value of Autopsy in Medical Education As Seen by Pathology 
Educators. Hill, R.B., & Anderson, R.E. (U.S.A.). (1991). Academic Medicine, 66, 97- 
100. 

A national meeting of pathology educators in 1989 provided the impetus for an 
exploration of new uses of autopsy in medical education. A month before the 
conference, the authors sent a questionnaire about the uses and value of autopsy in 
medical education to 120 persons registered to attend the conference. They used the 93 
responses, representing 69 U.S. and Canadian medical schools, as the basis of a 
workshop on the place of autopsy in future medical education. The present article is a 
report of the authors' findings from the questionnaire and workschop. They found that 
the uses of autopsy go far bey ond the traditional uses in teaching clinical pathophysiology, 
clinicopathologic correlations, clinical anatomy, gross and microscopic anatomy of 
disease, and visual skills. Emphasis was placed on the potential role of autopsy in 
education regarding legal/judicial proceedings, vital statistics, epidemiologic 
investigations, and public health, and in the understanding of such complex matters as 
medical fallibility, medical uncertainty, and grief. These purposes were seen as 
congruent with current societal concerns about the need to reverse the trend toward 
dehumanisation of medicine and physicians. The inability to realise these aims in the 
face of a precipitous drop in the autopsy rate is discussed. 
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Distance Learning in a Local Setting: A Structured Learning Course for the 
Introduction of General Practice to Undergraduate Students. Kamien, M., Macadam, 
D., & Grant, J. (Australia). (1991). Medical Teacher, 13, 353-361. 
This paper describes the development and evaluation of a structured introductory 
course in general practice. Following some of the principles developed in distance 
education, the Department provides everything the student needs for the formal 
learning requirement as well as detailed assistance in how best to tackle the selected 
topics. The course was reported to be demanding, relevant and enjoyable. The major 
areas requiring attention were in reducing the amount of reading required, help in 
learning to work in small student run groups and more one-to-one supervision of 
physical examination skills. With further refinement the course should be applicable to 
other medical schools in developed countries. 

How to Produce Instructional Text for a Medical Audience. McLeod, P.J. (Canada). 
(1991). Medical Teacher, 13, 135-144. 

Instructional text continues to play a major role in medical education. Production of this 
type of learning resource requires an understanding of the objectives of the learner and 
the characteristics of text which facilitate student learning. Success in producing 
effective readable text requires attention to the content as well as to the way it is 
presented. This review outlines the factors which have a major impact on those two 
elements. 
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Lecture Handouts of Projected Slides in a Medical course. Amato, S., & Quirt, I. 
(Canada). (1990). Medical Teacher, 12, no.3/4. 

Part of the teaching of clinical haematology to our third year medical students involves 
lectures given to the entire class. For the past several years, we have provided, at the 
beginning of each lecture, handouts on which are reproduced all or most of the 35-mm 
slides that will be projected during that lecture. The frames are positioned vertically on 
the left since of each page, thus allowing the students to add notes on the right side. 
There are advantages and disadvantages to this pedagogical method, but we believe that 
the former outweigh the latter. 



Evaluation 

The Effect of Compulsory Participation of Medical Students in Problem-Based 
Learning. Moore, G.T. (U.S.A.). (1991). Medical Education, 25, 140-143. 
Problem-based learning (PBL) is an instructional method that has attracted many 
advocates since its introduction in medical education almost 20 years ago. PBL features 
the use of student-directed tutorials, medically relevant problems to set study objectives, 
and independent learning. Educators have worried that not all students will do well with 
this method. This study compared a group of students who had chosen to be in a PBL 
curriculum with a group who had not, as they undertook a curriculum that contained 
both PBL and lecture-based courses. 

Academic performance was virtually identical regardless of learning method. Students 
slightly favoured the courses that featured the method they had originally chosen, but 
significantly larger proportion of students shifted their preference from the lecture to 
the problem-based approach than vice versa. Students' academic performance does not 
appear to suffer when they are involuntarily enrolled in a PBL curriculum and many 
come to prefer this type of curriculum. 

Effects of Conventional and Problem-Based Medical Curricula on Problem Solving. 

Patel, V.L., Groen, G.J., & Norman, G.R. (Canada). (1991). Academic Medicine, 66, 
380-389. 

This study examined the reasoning processes of beginning, intermediate, and senior 
students in two medical schools with different auricular formats. One school had a 
conventional curriculum (CC) where basic science was taught one and a half years 
before the clinical training, and the other had a problem-based learning curriculum 
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(PBLC) where basic science was taught in the context of clinical problems and general 
problem-solving heuristics were specifically taught. The students were asked to give 
diagnostic explanations of a clinical case, both before and after being exposed to 
relevant basic science information. Two distinct modes of reasoning were identified, 
each reflecting a curriculum type. A predominantly "backward-directed" hypothetic*)- 
deductive mode of reasoning was found in the explanations of the PBLC students, and 
a "more forward-directed" pattern of reasoning was found in the explanations of the CC 
students. Students in the PBLC produced extensive elaborations using relevant 
biomedical information, which was relatively absent from the CC students' explanations. 
However, these elaborations were accompanied by a tendency to generate errors. These 
results have important implications regarding the strengths and weaknesses of the two 
types of curricula. 
References 

Chi, M.T.H., Bassok, M., Lewis, M.W., Reiman, P., & Glaser, R. (1989). Self- 
explanations: How students study and use examples of learning to solve problems. 
Cognitive Sciences, 13, 145-182. 

Clancey, J.G. (1989). The knowledge level reinterpreted: Modeling how systems 
interact. Mach. Learning, 4, 287-293. 

Cole, C.R. (1990). Elaborated learning undergraduate medical education. Medical 
Education, 24, 14-22. 

Elstein, A.S., Shulman, L.S., & Sprafka S.A. (1978). Medicc 1 problem solving. An 
analysis of clinical reasoning. Cambridge, MA: Harvard University Press. 

Flexner, A. (1910). Medical education in the United States and Canada. A report to the 
Carnegie Foundation for the Advancement of Teaching. Bulletin no. 4. Boston, 
MA: Updyke. 

Maheux, B., Beland, R, Pineault, R., Rivest, P., & Valois, P. (1988). Do conventional 
and innovative medical schools recruit different students? Medical Education, 23, 
30-38. 

Norman, G.R., Tugwell, P., Feightner, J.W., Muzzin, LJ., & Jacoby, L.L. (1985). 

Knowledge and clinical problem-solving. Medical Education, 19, 344-356. 
Patel, V.L., & Groen, G.J. (1986). Knowledge-based solution strategies in medical 

reasoning. Cognitive Sciences, 10, 91-116. 
Patel, V.L., & Groen, G.J. (in press). Reasoning and instruction in medical curricula. 
Patel, V.L., Evans, D.A., & Groen, G.J. (1989). Biomedical knowledge and clinical 

reasoning. In: D.A. Evans, V.L. Patel, (Eds.). Cognitive science in medicine: 

Biomedical modeling. Cambridge, MA: The MIT Press. 
Patel, V.L., Evans, D.A., & Groen, G.J. (1989). On reconciling basic sciences and 

clinical reasoning. Teaching and Learning in Medicine, 1, 116-121. 
Patel, V.L., Evans, D.A., & Kaufman, D.R. (1990). Reasoning strategies and use of 

biomedical knowledge by students. Medical Education, 24, 129-136. 
Patel, V.L., Groen, G.J., & Arocha, J.F. (1990). Medical expertise as a function of task 

difficulty. Memory and Cognition, 18, 394-406. 



352 



366 



Patel, V.L., Groen, G J., & Frederiksen, C.H. (1986)- Differences between students and 
physicians in memory for clinical cases. Medical Education, 20, 3-9. 

Patel, V.L., Gtoen, G J., & Scott, H.S. (1988). Biomedical knowledge in explanations 
of clinical problems by medical students. Medical Education, 22, 398-406. 

Perkins, D.N., & Salomon, G. (1989). Are cognitive skills context-bound? Educational 
Research, 18, 16-25. 

Reder, L.M., Chamey, D.H., & Morgan, K.L (1986). The role of elaborations in 

learning a skill from an instructional text Memory and Cognition, 14, 64-78. 
Schmidt, H.G., Dauphinee, D.W., & Patel, V.L. (1987). Comparing the effects of 

problem-based and conventional curricula in an international sample. Journal of 

Medical Education, 52, 306-315. 
Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. 

Cognitive Science, 12, 257-285. 
Sweller, J., Mawer, R.F., & Ward, R.W. (1983). Development of expertise in 

mathematical problem solving. Journal of Experimental Psychology, 112, 639-661. 
Van Dijk, T.A., & Kintsch, W. (1983). Strategies of discourse comprehension. New 

York, NY: Academic Press. 
Walton, H.J., & Matthews, M.B. (1989). Essentials of problem-based learning. 

Medical Education, 23, 542-558. 

Effects of Tutors with Subject expertise on the Problem-Based Tutorial Process. 
Silver, M., & Wilkerson, L. (U.S.A.). (1991). Academic Medicine, 66, 298-300. 
With the increased interest in problem-based, small-group learning in medical education, 
a debate has arisen about whether the tutor should be an expert in the subject under 
discussion. This 1988 study at Harvard Medical School demonstrates that tutors' 
expertise has important effects on the process of discussion in a problem-based tutorial. 
In comparing discussions of subjects in which the tutors described themselves as expert 
with those in which they did not, the authors found that the tutors with expertise tended 
to take a more directive role in tutorials: they spoke more often and for longer periods, 
provided more direct answers to the students' questions, and suggested more of the 
topics for discussion. Tutor-to-student exchanges predominated, with less student-to- 
student discussion. These effects endanger an important goal of problem-based 
learning: the development of students' skills in active, self-directed learning. 

Comparison of Faculty Members* and Students' Perceptions Concerning Performance 
Criteria and Evaluation Strategies at the University of New Mexico School of 
Medicine. Conran, P.B., Obenshain, S.S., & Anderson, R.E. (U.S.A.). ( 199 1). Academic 
Medicine, 66, 553-557. 

Faculty members* and students* perceptions concerning the relative importances of (1) 
various performance criteria for students and (2) strategiesof evaluation weredetermined 
in 1989 by a questionnaire in a single medical school that had both a problem-based, 
student-centered curriculum and an organ-t?lock curriculum (i.e., regular track)* The 
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greatest differences in the rankings of both areas were found between those given by 
the freshman and sophomore students in the problem-based curriculum and those given 
by the same groups in the regular track. The faculty members' perceptions tended to be 
closer to those of the students in the regular track. 

Continuing Medical Education and Career Choice Among Graduates of Problem- 
Based and Traditional Curricula. Tolnai, S. (Canada). (1991). Medical Education, 
25, 414-420. 

A survey was conducted among graduates of two Canadian medical schools who have 
been in practice for more than 9 years. The purpose of the study was to test the 
hypothesis that graduates of a problem-based curriculum differ from graduates of a 
traditional curriculum in their attitude to and participation in continuing medical 
education (CME) activities. Differences were noted in the rate of participation in 
certain CME activities (attendance at national and international conferences and 
meetings) between specialists and family doctors in both groups of alumni. However, 
the data indicate that the differences in learning-teaching methods employed in the 
course of the undergraduate medical curriculum do not exert a decisive influence upon 
the learning habits of the graduates. 
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Problem-Based Learning 

Problem-Based Learning in a Surgery Clerkship. Blosser, A., & Jones, B. (U.S.A.). 
(1991), Medical Teacher, 13, 289-293. 

The authors were member of the first group of medical students to participate in a newly 
modified third-year surgery clerkship at the University of Kentucky College of 
Medicine. The primary teaching methodology of this clerkship is problem-based 
learning (PBL). In this type of learning experience, students work with paper cases as 
though they were actual patients, using a method similar to that which they will later 
use as residents or practising physicians. The students make the decisions themselves, 
formulating differential diagnoses, eliciting relevant items from a history and physical 
examination, proceeding with a diagnostic work-up, and creating a treatment plan. This 
problem-based method increases students 9 use of resources, improves their retention 
of information, and helps them develop time management skills. Although the students 
believe that the clerkship as a whole should offer more opportunities for hands-on 
experience, they found the PBL component of the clerkship highly motivating, 
intellectually stimulating, and experientially satisfying. 

Commentary on" Problem -Based Learning in a Surgery Clerkship". Schwartz, R.W. 
(U.S.A.). (1991). Medical Teacher, 13, 295-298. 

The Department of Surgery at the University of Kentucky College of Medicine is 
developing a modified third-year clerkship which stresses active student involvement 
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in the educational process, the stimulation of iife-long intellectual curiosity, and the 
development of problem-solving and time-management skills. The cornerstone of this 
programme is problem-based learning (PBL). PBL places students in a situation in 
which they must identify the major learning issues and solve the clinical problems 
presented in a series of paper cases. The students learn and practice a problem-solving 
method which rekindles their intellectual excitement for medical knowledge. After 
completing this programme, the students are prepared and stimulated to be immersed 9 
in the clinical setting through acting internships. The first students to complete the 
programme have articulated concerns about the mixture of traditional clerkship clinical 
activities and PBL. These concerns are being addressed in subsequent modifications of 
the clerkship. 

Student-Directed Problem-Based Learning in General Practice and Public Health 
Medicine. Usherwood, T., Joesbury, K., & Hannay, D. (United Kingdom). (1991). 
Medical Education, 25, 421-429. 

The development and implementation are described of a new 6-week course in general 
practice and public health medicine for final-year medical students. This course is 
based on the principles of student-directed problem-based learning in small groups and 
makes substantial use of student attachments to local general practices which act as 
learning resources. Student assessment is by profiles. Course evaluation is by qualitative 
feedback, and the results of this are presented. The course offers a flexible learning 
environment in which the aims of its designers and the goals of their students can be 
achieved. 

The Construction of a 'Topic Tree 9 : A Way of Familiarising a Teaching Staff to 
Problem-Oriented Learning in a Master's Programme in Public Health. Chastonay, 
P., Guilbert, J.J., & Rougemont, A. (Switzerland). (199 1). Medical Education, 25. 405- 
413. 

At the planning stage of a community-oriented and problem-based learning master's 
programme in public health it is indispensable to build a network of teachers, experts 
in specific fields, willing to guide the students in acquiring new competencies and ready 
to facilitate the experimental learning process. We di scuss in this paper the construction 
and the utilisation of the 'topic tree 9 . In addition we show how the teaching staff get 
acquainted with that educational methodology by experimenting with it themselves 
(prior to the students). About 10 experts, each one a specialist in given fields, have 
conceptualised their approach in building a topic tree (concept tree); such a tree makes 
it possible to schematise the topics to be understood in order to solve a given problem 
and the relationship existing between actions to be undertaken. The experiment, meant 
to initiate the teaching staff in experimental learning procedures, was appreciated by 
all the experts, who considered it a creative and stimulating method. It also permitted 
the construction of a bibliography including key documents concerning each identified 
priority health problem. We discuss the utility and the relevance of our approach in the 
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perspective of initiating teachers in experiential learning based on problem-solving. 



Problem-Solving 

Formal Decision Supports in Medical Practice and Education. Christensen, C, Elstein, 
A.S., Bernstein, L.M., & Balla, I. (U.S.A., Australia) (1991). Teaching and Learning 
in Medicine, 3, 62-70. 

Several formal, quantitative techniques have been developed to help physicians make 
diagnostic and treatment decisions. Despite the precision of these models, as well as 
growing documentation of systematic errors in intuitive reasoning, most clinicians 
choose not to utilise decision supports. This may be due to weaknesses inherent in 
formal models, practical problems implementing them in the clinical environment, or 
psychological discomfort that arises when physicians try to quantify diagnostic and 
therapeutic uncertainty or patients' preferences. An examination of these difficulties 
however does not reveal an adequate basis for rejecting formal decision techniques. 
Quantitative decision aids can have a positive impact on clinical reasoning and can 
provide a valuable educational tool for structuring clinical problems. These models can 
be used to help teach students how to select, structure, and process clinical information. 
Medical educators now have available models that will enable them to teach several 
aspects of clinical reasoning rather than rely solely on practice and experience for the 
development of this important skill. 
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Medical Problem-Solving: An Exploration of Strategies. Ridderikhoff, J. (The 
Netherlands). (1991). Medical Education, 25, 196-207. 

In contrast to factual knowledge the reasoning processes of doctors in their attempts to 
clarify the patient's problem has been a somewhat neglected domain of study. The rise 
of experimental psychology, clinical decision analysis, and problem-based learning 
fostered studies in this area. Several studies from different viewpoints have been 
performed, leading to a more profound understanding of these processes. I approached 
the issue from a general physician viewpoint which reverted to the old distinction 
between deductive and inductive reasoning. Within a group of 68 participating doctors 
(family doctors and general physicians) the inductive method was exclusively employed 
From this finding we conjecture that the inductive type of reasoning is the predominant 
style in the medical world. The consequences as attached to the overall utilisation of the 
inductive method are far-reaching. Among other, process retracing as a feedback 
mechanism fails to function when it results from the intuitive nature of the (pattern- 
recognition) process. As a consequence we do not and can not know what we may learn 
from experience, good things as well as bad things. The inductive method is a method 
of practice and quick reaction, but it leaves us empty-handed as far as understanding 
and teaching are concerned. 
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Teaching Medical Decision Making and Students 9 Clinical Problem Solving Skills. 

Rogers, J.C., Swee, D.E., & Ullian, J.A. (U.S.A.). (1991). Medical Teacher, 13, 157- 
164. 

Medical students need to be taught explicitly about decision making to be prepared for 
the changing health care environment. Medical decision making curricula have 
received favourable responses from students and have influenced some aspects of 
student performance. Questions remain about the impact of the teaching on students' 
general problem solving skills. A 15 hour course covering decision making topics was 
presented during a preclinical elective prcceptorship for 5 years. Problem solving 
ratings made by clinical supervisors for the third year psychiatry and internal medicine 
clerkships were not better for the students who had the instruction and clinical 
experience than for the students in the comparison group. The results suggest that this 
approach to teaching decision making requires further i jvelopment and testing. 



Skills Training 

Psychomotor Skills for the General Professional Education of the Physician. Irby , D.M., 
Lippertin,F.G.,&Schaad,D.C. (U.S.A.). (1991). Teaching and Learning in Medicine, 
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3, 2-5. 

Although concern about the clinical skills of medical school graduates is increasing, 
there is little consensus among faculty about what the essential clinical skills are for the 
general professional education of the physician. To identify such a common core of 
psychomotor skills, a survey questionnaire was mailed to faculty at the University of 
Washington School of Medicine. Fourty-three skills were determined to be essential for 
mastery by more than 60% of respondents. These results were then validated by a 
committee of clerkship directors who refined the list to 28 skills that should be required 
for graduation. These data are being used to specify educational objectives and to 
design clinical performance examinations. 

Training Medical Practitioners in Information Transfer Skills: The New Challenge. 

Sanson-Fisher, R.W., Redman, S., Walsh, R., Mitchell, K., Reid, A.L.A., & Perkins, 
J.jf. (Australia). (1991). Medical Education, 25, 322-333. 

Traditionally, undergraduate medical education has concentrated on teaching students 
how to gather information or take medical histories from their patients. However, 
research increasingly indicates that there is a need for medical practitioners to improve 
their skills in information transfer in a way which will increase the probability that 
patients are active collaborators in their treatment. Consequently, Newcastle Medical 
School has sought to develop training packages for medical students in information 
transfer skills. This paper describes the resulting training programme with particular 
emphasis on the areas selected for training, the methods by which students are taught, 
the necessary interactional skills and the assessment procedures which are applied. 
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To what Extent are Medical Interviewing Skills Teachable? Kraan, H.F., Crijnen, A.M., 
De Vries, M.W., Zuidweg, J., Imbos, T., & Van der Vleuten, CP. (The Netherlands). 
(1990). 

Medical Teacher, Vol. 12, No. 3/4, 315. 

Growth patterns of medical interviewing skills during a 6-year undergraduate curriculum 
are assessed by studying 563 medical students taken from five year-groups, interviewing 
simulated patients. In a cross-sectional, quasi-experimental design their skills are rated 
by means of the Maastricht Histoiy-taking and Advice Checklist (M AAS), an observation 
instrument which measures five categories of interviewing skills pertaining to initial 
medical consultations. The findings suggest that the skills for "history-taking", 
"presenting solutions" and "structuring of the interview" are effectively learned. These 
learning effects result from a continuous small group teaching programme with expert 
and peer review of videotaped encounters with simulated patients. The teaching effects 
of this programme seem less for the skills pertinent to the phase of "exploring the 
reasons for encounter" and to the "basic interviewing skills", because the students* 
growing medical knowledge and the increasing ability to solve medical problems exert 
a counteracting influence on the acquisition of these easily deteriorating skills. The 
results might be helpful to curriculum planners in order to make their programmes for 
medical interviewing skills more effective. 

Effects of Communication Skills Training on Students 9 Diagnostic Efficiency. Evans, 
B.J., Stanley, R.O., Mestrovic, R., & Rose, L. (Australia). (1991). Medical Education, 
25, 517-526. 

This study evaluates the impact of a training programme in communications skills on 
subsequent diagnostic efficiency. Videotaped history-taking interviews conducted by 
groups of specially trained and control groups of students were rated for their diagnostic 
efficiency by two medical practitioners. Students in the trained group had shown 
greatly increased skills in interviewing and interpersonal effectiveness as a result of 
their training. 

A comparison of ratings given by the two experimentally naive, independent observers 
revealed that trained students were significantly better at eliciting full, relevant data 
from patients - they were diagnostically more efficient, but took no longer than their 



368 



control group counterparts to elicit the information 

Further research with the medical interview rating scale will clarify the skills required 
of medical students in interviewing and diagnosis and facilitate remedial training for 
students who show poor interview skills. 
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Students' Learning 

Approaches to Learning of Students in an Indonesian Medical School. Emilia, O., & 
Mulholland, H. (Indonesia). (1991). Medical Education, 25, 462-470. 
This paper reports on a study of approaches to learning ^f undergraduate medical 
students in the University of Gadjah Mada, Indonesia. The Lancaster Approaches to 
Studying Inventory was translated into Indonesian and the translated form pilot tested. 
The instrument was then completed by 90 students, 30 each in first, second and fifth 
year in the Faculty of Medicine. It was found that Indonesian students generally gave 
higher rates than previous studies have reported. Factor analysis of their responses 
showed strong resemblances to other groups in the factors of meaning and reproducing 
orientation. Differences found in Indonesian students' responses were in strategic and 
non-academic orientations. There were differnces between the response pattern of first- 
third- and fifth-year students. It is concluded that the instrument is valid for use in 
Indonesia. 
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In the first part of this study a comparison is made between the approaches to studying 
adopted by failing and achieving nursing college students. It is concluded that failing 
students manifest approaches to studying that are, in some respects, conceptually 
difficult to interpret. This conclusion is presented against a background of similar 
findings that have emanated from other recent studies on student learning. 
The second part of this study investigates the influence of contextual perceptions on the 
manifestation of the approaches to studying adopted by achieving students. In thus 
exploring the manifestation of different forms of contextualised approaches to studying 
(which are referred to as study orchestrations) a distinction is made between the 
influences attributable to "conventional" and "clinical" sets of contextual variables. In 
an extension of previously reported studies, it is concluded that the combines set of 
these contextual variables explains significantly more of the variation in study 
approaches than the set of "conventional" variables. It is further concluded that, under 
the influence of an extended set of contextual variables, conceptually related, but 
differing forms, of study orchestration may be manifested; a general form which is 
readily interpretable in terms associated with "conventional" academic contextual 
variables and a clinical form which, in the present study, is associated specifically with 
a nursing academic environment. The implications of these conclusions for nursing 
education are discussed and areas for future research are indicated. 
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Research 

Can Medical Education be Researched, Lloyd, D. A. (Canada). (1991). MedicalTeacher, 
13, no.2. 

Many medical educators are tempted or encouraged to conduct research into teaching 
techniques, methods of evaluation, and curricular change . These educators are generally 
career physicians who become distressed by their inability to apply the social science 
model of research to their project with impunity. This paper addresses the issues 
involved in conducting educational research and provides some suggestions for 
methodology that can be used. 
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The Potential of the Academic Medical Center to Shape Policy-Oriented Rural Health 
Research. Rosenblatt, R.A. (U.S.A.). (1991). Academic Medicine, 66, 662-667. 
Rural communities continue to have problems in gaining access to basic health care 
services, a problem exacerbated by persistent shortages of physicians, financially 
threatened rural hospitals, and weak local economics. Academic health centers can help 
to address the issues, not only by increasing the flow of their graduates to rural areas, 
but also by supporting health services research designed to shape public policy that 
effects the rural United States. Examples of such research include experiments 
designed to influence the locational decisions of medical students and residents, studies 
of the quality and cost-effectiveness of care in rural hospitals, and the testing of new 
ways to provide emergency medical care in rural areas. Such policy-oriented research 
is compatible with both the intellectual and the service missions of most medical 
schools; in addition, lessons learned in rural areas may be relevant in more urbanised 
areas. 



Teaching 

Retraining Faculty for the Problem-Based Curriculum at the University of Hawaii, 
1989-1991. McDermott, J.F., & Anderson, A.S. (Hawaii). (1991). Academic Medicine, 
66, 778-779. 

The problem-based learning (PBL) curriculum, adopted by and increasing number of 
medical schools, requires retraining faculty for new roles as tutors with small groups 
of medical students. This study describes the procedures and results of four PBL 
training workshops (1989-1991) given at the University of Hawaii John A. Bums 
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School of Medicine. Eighty-eight faculty volunteers each answered a 20-item multiple- 
choice questionnaire testing their knowledge of PBL both and after they participated 
in workshop. The post test result were used to identify deficiencies in the tutors 9 
knowledge and practice of both their role and their function. The authors conclude that 
these deficiencies, now identified, will be addressed in a systematic fashion in 
subsequent workshops. 

Characteristics of Effective Clinical Teachers of Ambulatory Care Medicine. Irby, 
D.M., Ramsey, P.G., Gillmore, G.M., & Schaad, D. (U.S.A.). (1991). 
Academic Medicine, 66, 54-55. 

This study identified characteristics of clinical teacher in ambulatory care settings that 
influenced ratings of overall teaching effectiveness and examined the impacts of 
selected variables of the clinic environment on teaching effectiveness ratings. A survey 
instrument derived from prior research and observations of ambulatory care teaching 
was sent to 165 senior medical students and 60 medicine residents at the University of 
Washington School of Medicine in 1988. A total of 122 (74%) of the seniors and 60 
(7 1 %) of the residents responded. Results indicate that the most important characteristics 
of the ambulatory care teacher were that they actively involved the learners, promoted 
learner autonomy and demonstrated patient care skills. Environmental variables did not 
have a substantial influence on these ratings. 

A Multidisciplinary Programme to Improve the TeachingSkills of Incoming 
Housestaff. Pristach, C. A., Donoghue, G.D., Sarkin, R., Warfula, C, Doerr, R., Opila, 
D., Stem, M., & Single, G. (U.S.A.). (1991). Academic Medicine, 66, 172-174. 
Residents at the School of Medicine and Biomedical Sciences at the State University 
of New York at Buffalo (SUNY at Buffalo) expressed an interest in receiving 
instruction in teaching skills. In 1988, under sponsorship of the Graduate Medical- 
Dental Education Consortium, faculty and residents implemented an orientation 
programme for 1 10 incoming house staff. Residents received instruction in large-group 

' teaching skills, bedside teaching, small-group discussion, and student evaluation. The 
programme was rated positively by the residents, 5% of whom became instructors in 
the orientation programme for the following year. Because of the programme's 
structure, both faculty and housestaff receive training in teaching skills, which may lead 
to an overall improvement in the teaching of SUNY at Buffalo's medical students. 
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NETWORK INFORMATION 
Full Members 

- Faculty of Medicine, University of Newcastle, Australia 

- College of Medicine and Medical Sciences, Arabian Gulf University, Bahrain 

- College of Health Sciences, Ministry of Health, Bahrain 

- Faculty of Health Sciences, McMaster University, Hamilton, Canada 

- Faculty of Medicine, University of Sherbrooke, Sherbrooke, Canada 

- Medical Faculty, Universidad de la Frontera, Temuco, Chile 

- Xian Medical University, Xian, China 

- Faculty of Medicine, Escuela Colombiana de Medicina, Bogota, Colombia 

- University of Havana, Havana, Cuba 

- Faculty of Medicine, Suez Canal University, Ismailia, Egypt 

- Jimma Institute of Health Sciences, Jimma, Ethiopia 

- Fiji School of Medicine, Suvai, Fiji 

- Faculty of Medicine, University of Helsinki, Helsinki, Finland 

- Medical Faculty, University of Tampere, Tampere, Finland 

- Medical School, University of Ghana, Accra, Ghana 

- School of Medical Sciences, University of Science and Technology, Kumasi, Ghana 

- Christian Medical School, Veliore, India 

- School of Medicine, Gadjah Mada University, Yogyakarta, Indonesia 

- College of Medicine, Tikrit University, Tikrit, Iraq 

- Faculty of Health Sciences, Ben-Gurion University of the Negev, Beer-Sheva, Israel 

- Medical School, University of Ban, Ban, Italy 

- I.C.H.M., Istituto Superiore di Sanitk, Rome, Italy 

- Faculty of Medicine, University of Turin, Italy 

- School of Medical Sciences, University Sains Malaysia, Kubang Kerian, Kelantan, 
Malaysia 

- Faculty of Health Sciences, Universidad Aut6noma Metropolitan Xochimilco, Mexico 

• School of Health Sciences, Universidad Aut6noma de Nuevo Le6n, Nuevo Le6n, 
Mdxico 

• Institute of Medicine, Tribhuvan University, Kathmandu, Nepal 

- Faculty of Health Sciences, University of Limburg, Maastricht, 'Tie Netherlands 

• Faculty of Medicine, University of Limburg, Maastricht, The Netherlands 

• Faculty of Clinical Sciences, Bayero University, Kano, Nigeria 

- Faculty of Health Sciences, Obafemi Awolowo University, Ile-Ife, Nigeria 

• College of Medicine, Ogun State University, Sagamu, Ogun State, Nigera 

• Faculty of Health Sciences, University of Ilorin, Bonn, Nigeria 

- College of Medical Sciences, University of Maiduguri, Maiduguri, Nigeria 

- School of Medicine, University of Troms0, Troms0, Norway 
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- Faculty of Health Sciences, Aga Khan University, Karachi, Pakistan 

- College of Medicine, University of The Philippines, Manila, The Philippines 

- College of Medicine and Health Sciences, Omdurman Islamic Univ., Omdurman, Sudan 

- School of Medicine, University of Gezira, Wad Medani, Sudan 

- Faculty of Medicine, Health University, Linkdping, Sweden 

- Faculty of Medicine, Chiang Mai University, Chiang Mai, Thailand 

- Faculty of Medicine, Chuialongkorn University, Bangkok* Thailand 

- Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand 

- Faculty of Medicine, Prince of Songkhla University, Songkhla, Thailand 

- Faculty of Medicine, Makerere University, Kampala, Uganda 

- Faculty of Medicine, Brown University, Providence, U.S.A. 

- School of Medicine, Mercer University, Macon, U.S.A. 

- College of Human Medicine, Michigan State University, East Lansing, U.S.A. 

- College of Osteopathic Medicine, Michigan State University, East Lansing, U.S.A. 

- School of Medicine, Southern Illinois University, Springfield, U.S.A. 

- College of Medicine at Rockford, University of Illinois, Rockford, U.S.A. 

- School of Medicine, University of New Mexico, Albuquerque, U.S.A. 

- School of Medicine, University of North Carolina, Chapel Hill, U.S.A. 

- School of Medicine, Wright State University, Dayton, U.S.A. 

- Faculty of Medicine, Universidad de la Reptiblica, Montevideo, Uruguay 

- School of Medicine, University of Zambia, Lusaka, Zambia 
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University, Ismailia, Egypt 
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The Network of Community-Oriented Educational Institutions 
for Health Sciences 

General Aim 

The general aim of the Network is to provide mutual support to member institutions who 
wish to adapt their curricula to the health needs of the communities which they serve. 

Main Objectives 

There are five primary objectives: 

• strengthening of membership institutions in their implementation of 
community-oriented learning and appropriate instructional methods; 

• strengthening of faculty competences related to community-based 
education; 

• developing techniques, approaches, methods and tools appropriate to a community- 
oriented curriculum, e.g. problem-based learning; 

• promoting population concepts in the health services system and the curriculum; 

• assisting institutions in countries that have decided to introduce innovations in the 
training of health personnel, with the ultimate goal to improve health care and to 
contribute to the achievement of "Health for All". 

To achieve these objectives, several additional objectives have been identified: 

• Facilitating the development of curricula for health professions which help students 
become competent in the solution of health problems of communities, as well as of 
individuals and families in a community context. 

• Problem-based learning is seen as a powerful educational strategy to maximise 
students' acquisition of relevant knowledge, skills and attitudes. Specific considera- 
tions include: 

- the systematic selection of problems and population based concepts which represent 
the health needs of the community; 

- the definition of relevant skills, including skills in problem-solving, independent 
learning, critical appraisal of evidence, and teamwork; 

- the strengthening of teaching capability of staff related to community-orientation 
and problem-based learning; 

- the development of methods for curriculum design and learning tools appropriate 
to a community- and problem-based curriculum; 

- the design of procedures and tools for the assessment of student and graduate 
performance; 

- the evaluation of innovative programmes including their contribution to the develop- 
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rnent of effective, efficient and humane health care, and the commitment to "Health 
for All by the Year 2000", as promoted by the World Health Organization. 

• Implement health research programmes that include basic, applied and operational 
research that is relevant to health and health care problems in the community served 
by the institution. Particular attention is given to strengthening health systems and 
epidemiologic research. 

• Establishing relationships with the health care services and health care delivery. This 
link is intended to promote coordination between health care services, health man- 
power development and the promotion of primary care. 

• Describing and developing organisational and managament strategies which maxi- 
mise the achievement gf institutional goals within the community setting. 

The Network sees community-orientaion and problem-based learning as specific issues 
which merit particular emphasis at tb*5 present phase of evolution of education in the health 
sciences. As the needs for health care and the further development of the health sciences 
and practices change, the Network will reconsider its priorities in the education of health 
professionals. 
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Membership of the Network 

Since 1979 the membership has grown to a total of 56 full member institutions, 93 
associate member institutions and 44 corresponding members in 1 992. There are four types 
of membership: 

1 . Full membership can be acquired by educational institutions for health sciences which 
implement community-oriented education and which are willing to collaborate with 
other institutions in achieving the goals of the Network. Full members have voting 
rights. 

2. Associate membership can be acquired by institutions, organisations or groups of 
people which are interested in the objectives of the Network and the activities that flow 
from them and who wish to play an active part in them. Associate membership can also 
be acquired with a view to the acquisition of full membership at a later stage. Associate 
members may not vote. 

3. Corresponding membership can be acquired by individuals who are interested in the 
objectives of the Network. Corresponding members may not vote. 

4. Honorary membership will be granted to individuals who have rendered exceptional 
service to the Network. Honorary members may not vote. 



Enquiries should be addressed to Pauline M.J. Vluggen, Network Secretariat, P.O.B. 616, 
6200 MD Maastricht, The Netherlands. 



